DOCUMENT* RESUME' 



ED 233 *82 

AUTHOR 
TITLE 



^INSTITUTION-' 

REPORT NO 
. PUB, DATE ; 
NOTE-.; 
PUB, TYPJS' 

EDRS -PRICE 
.DESCRIPTORS 



GEj: 036 670 



Evans, Ridhard M. ; Braby> Richard ' 
Self^Pacedt ^hd' Convent ional I nst ruct ion in Navy 
Training: A Comparison on Elements of Quality. 
Technical Report 147. - , t 

Ntfval Training Analysis and Evaluation Group, 

Vftrlando, Fla. * - 

TAEG-'TR-147 . 

- Jul" 83 ' 

; 19p. : ■ ~. *- ; - 

Reports - Research/Technical (143) 



) 



IDENTIFIERS 



ABSTRACT 



MF01/PC04 Plu§ Postage." ; " - ' 

Administrator Attitudes; . Comparative Analysis,-, 
Computer Managed Instruction; Cues; *Educatioftal 
Quality; feedback; * Individualized Instruct ion ; . s 
Military Personnel ; *Mi^itary- Training;^ *Pacing; 
Participation; Prerequisites? Program, Casts; 
Questionnaires; Reinforcement; Student Attitude^ 
StuSent Participation; Surveys; Teacher Attitudes; * 
♦Teaching Methods'; Technical ^Education; *Time Factors 
(Learning) , 

Marine Corps; Jlavy; *Self Paced Instruction 



A study compared the approaches of self -paced and 
conventional- instruction in Navy training'. Using a sample of 37> Navy 
and Marine Corps courses, researchers contrasted the conventional 
mixed, computer-managed, and self-paced instructional approaches 
according to the following six measures of instructional quality: 
prerequisites, cues, ^participation, reinforcement, feedback, and : * 
correctives. Various "data collection strategies were employed during 
r the study, including site visits; analysis of instructional 
'materials; and questionnaires administered to students / 'instructors, , 
and supervisors. Based on data from all of these sources, the 
researchers concluded that the six quality-of-instwifction elements 
appear to ..occur with * more* frequency in conventional rather than in 
mixed or self-paced instructional courses. Both student and 
n on student direct cpsts for the self-paced courses examined were from 
36 -.tb.v5£ percent of those required for the conventionally taught---^ 
. courses i :.! Because self-paced instruction w^s found to be heavily / 
depended i bri ; ^£ten instructional materials, researchers -Urged 
particular Cji^p^xh^e design of instructional materials a^d 
objectives for self-paced courses* Since instructional strategy did 
not ultimate^ be a determining factor in the .overall 

effectiveness of current Navy courses, researchers suggested that 
choice of /instructional strategy rest primarily on the relative cost 
effectiveness of the) approaches. (MN) 



********************************** 

* Reproductions supplied by EDRS are the bpst that canbe made 

* L •« from the original document." 

********************** ******************************** **************** 



GO 



Technical Report 147 



SE.LF-PACEDW\NP: CONVENTIONAL INSTRUCTION IN NAVY TRAINING: 



A 'COMPARISON ON ELEMENTS OF QUALITY 



vRic,har'd M". Eva/is 
Richard Braby 



Training Analysis ftnd Evaluation Group 



' July 1983 



US. DEPARTMENT OF EDUCATION 

NATIONAL INSTITUTE OF EOUCATION - 
iDUCATIONAL RE SOURCE ^INFORMATION 
/ " CENTER (ERIC) * 

f This "tfbcument has been rqpro'duced as 

received from the person or organization 

originating it.. 
I Minor changes have been made to improve 

reproduction quality. 

; \ — 

Points of view or opinions stflted in this docu- 
ment do not necessarily represent official ME 
position or policy. 



, r 




GOVERNMENT RIGHTS IN DATA> STATEMENT 

Reproduction of this publicAtiort in whole 
or in part is permitted for any purpose 
of the .Uni ted States Government . 



ALFRED F.' SMODE, Ph.D., Director. 
iTraining Analysis and •Evaluation Group 




W. L..MAL0Y, Ed.D. 
Principal Civilian Advisor 
on Education and Training 



ERIC 



Unclassified 



SECJRITY CLASSIFICATION OF THIS PACE (Wh9n Dete Entered) 



V \ REPORT DOCUMENTATION PAGE' 


read instructions 
" Before completing form , 


1. REPORT HUMBER 2. GOVT ACCESSION NO. 

Technical Report 147 


3. RECIPIENT'S CATALOG NUMBER -* J 


4. TITL c ( Ana Subtitle) 

SELF-PACED AND CONVENTIONAL * 

INSTRUCTION IN -NAVY TRAINING: 

A COMPARISON ON ELEMENTS OF QUALITY 


S. TYPE. OF REPORT 4 PERIOD COVERED 


6. PERFORMING ORG. REPORTS^UMBER 


7; authors; 

Richard 6 M. Evans and Richard Braby 


8. CONTRACT OR GRANT NUMBER(»J 
f 


9. PERFORMING ORGANIZATION NAME AND ADDRESS 

Training\ Analysis and Evaluation .Group, 

Department of the foavy f 

Orlando, \FL - 32813 i " ■■ ' ' 


10. PROGRAM ELEMENT, PROJECT, TASK $ 
AREA & WORK UNIT NUMBERS 

*" 1 


11. CONTROLLING OFFICE NAME AND ADDRESS 

. 'V • 


12. REPORT DATE 

Julv 1983 > 


13. ' NUMBER OF. PAGES 

7ft. ; '.' 


'14. MONITORING AGENCY NAME A A'OO R FSVI/ Hlffnrmnt from Cnntrallina Hffirm\ 

r ' 

\ - ■ . - ■ 

\ ,. r ... 


■ s. 3tUU ni 1 Y \Ol. *nta tOpOTZj 

Unclassified 


15*. DECLASSIFICATION/ DOWNGRADING 
SCHEDULE 



16. DISTRIBUTION STATEMENT (of thie Report) 

'Approved for public release; .distribution is unlimited. 



7. DISTRIBUTION STATEMENT (of the abetrqct entered in Block 20, it different from Report) 



8. SUPPLEMENTARY NOTES 



9. KEY WORDS (Continue on reveree eide it neceeemry mnd identify by block number) 



^Self-Paced 
Individualized 
Quality of Instruction 
Prerequisites 



Cues 
Feedback 
Participation 
Reinforcement 



Correctives 
Direct Costs ; 
Time to Mastery" 
.Navy Technical Training 



20. ABSTRACT (Continue on reveree tide if. neceeemry mnd identify by block number) 

This study examined a sample of 37 Navy and Marine Corps courses 
purporting some degree of individualization in instructional method.* 4 
Courses were assigned to three categories for analysis:- conventional, 
mixed, and self-paced instruction. Data were collected from site vistts, 
samples of instructional- materials , and questionnaires administered to 
students, instructors, and supervisors.* % (continued on reverse) 



ERLC 



DD 



FORM 
1 JAN 73 



1473 



Unclassified - 



' . f 

EDITION Of 1-NOV 65 IS OBSOLETE ' 

5/N 0 1 02- LF- 01 4- 6601 , — — 1 / 

f » srr- SECURITY CLASSIFICATION OF THIS PAGE (When Dete Bntered)! 

3 



x 



Unclassified 



SECURITY CC^SSIRCATION'OF THIS 



E fWiMi Dmtm Entmred) 



(20) . Abstract ( Continued) " » . 

Contrasts among categories were made on six measures of instructional 
quality: ' prerequisites,. cues, participation, reinforcement, feedback,' and 
corrective's: fjrie three categories of courses differed significantly on each 
of the '.measures! .■•*;.. . \ 

Al thoughV.pef'cfeptions j pf- ^uesttohnaire respondents were generally 
favorable concerning their courses, it was concluded that ideal . •■ 
individualized 4 nstruction'- rarely occurs -in the NAVEDTRACOM. While 
conventional courses: were' often found to' be higher in measures of quality 
instruction, their 'costs weifee .also higher; 1 Recommendations for changing 
existing training directives -are made.. ■ ' ^ 




i 



S N.0f02- LF- 014- 6601 



Unclassified 



' SECURITY CLASSIFICATION OF THIS PkGt(Whit £>•<• Bntmtid) 
4 „. 



'. . ' ~ Technical Report 147 • 

. . / TABLE OF CONTENTS- ; " • ." 

Section . < Page 



INTRODUCTION, 



Purpose of the. Study. .V Jj\'. 5 . 

- Def initions. . ... -6 

Conventional .Instruction...., ......... i. m 6 

' Mixed Instruction^. . .............. 6 

Sel^-Paced Instruction...... 6 

Individual iz£d Instruction. . ... . . . ; . . * 

' Qual it^of Instruction^,. . . . i . :->■."*> v w. v .vv.*./. ;;6 

• ■ '. "■ .■ ; t v«- 

■v '* Prerec|u i s i tes .......... ........... ....f. ........ r 6.'- 

• ..x • Cues.;>. 7 

" . Participation.......'. *. ............... 7 

■ ' ••' ' Reinforcement..:.*.* .. . <7u 

A , Feedbatlci...' v ., 7 

. > Corfectlves*.;^. . , 7 

Plan of. the Study. ............ J.. ...... 77... , 8 

Organization of the Report.'. ...'J.....,..' .; 8' 

II (J\^ TECHNICAL' APPROACH. ... . . . . . .' 9 

Description of the Sample . . 9, 

- Description of the Categories oflll ... 11 

Quality of Instruction Questionnaire,. , 15 

Time to Mastery Analysis....... ^ 16 

Structured^ Interview - 17 

Evaluation of NAVEDTRA 110- (Series) Guidelines 17 

III RESULTS ... 18 

Quality of instruction .............. 18 

Time to Mastery Analysis v ^ 22 

Cost Analysis .... .. 22 

Structured Interview... k '..-. 24 

' m ] Evaluation of Draft NAVEDTRA, 110B Guidelines 27 

IV - DISCUSSION...:....... . 29 

V CONCLUSIONS AND RECOMMENDATIONS. T 32 

'> * Conclusions ; . 32 

v "Recommendations 33 



ERIC 



Technical Report 147 
' TABLE OF CONTENTS (continued) 



Section * „ 
BIBt^OgRAPHY. 



APPENDIX. A :■ Course Description Form with Summary- Statistics. • . . . . 

APPENDIX,, B Structured Interview for Evaluating Instructional 
f . Materials........ 



Page 
37 

;. 40 

48 



APPENDIX. C Quality of Instruction Questionnaire 54 

AP PEN DfX D Mean Values and Significance of ThVee VARII on 

Quality of Instruction Questionnaire. ;. 61 



APPENDIX E Review of the Literature. 



64 



Historical 65 

Theoretical : 65 

QI Variables. 68 

Costs......... 70 

Summary . ..... » 71 

> ■ 

APPENDIX F A Model of Individualized Instruction..'. -. . 72 



ERIC 



Technical Report 147 



Figure 

*1. 



3 



Table 



.LIST OF ILLUSTRATIONS " - ' '* ' 

Three VARII .by Six -Q I . -. : . ? m \ 20 



2 ^o fil e s of 1,6^3 Students, 164 Instructors, and . 

52 f Supervisors on ft Si x QI Vari abl es k . ....... ".. ....... f 21 



Coefficients of Variation .in Module Time to Masterv 
in Four Self-Paced Courses. *." ' 



-Paced Courses...!.."...'. ... i4 . .... 23' 

E-l Major Variables in the Theory of School Learning^.......;. 67 

« * 

F-l' A Model of Quality Individualized Instruction.... ' 74 



LIST OF TABLES 



. . • J s - Page 

1 Population of NITRAS Courses by Location ... .10 

2 Course, Population. 



..... 12 

3 ■ Mean Class Time and Criterion to Mastery for 

Three VARII... 14 

• >■ . . l . ■ "", S" 

4 Percent Academic Day Spent -in Three VARII by 

Seven Learning Center Activities...... . 14 ■ 

5' Percent Classroom Self-Study Time in Three Levels of * 

Individualization by Six Types of Study Materials 14 

6 Mean Number of Personnel in Classroom or < . 

Learning Center '. _ j g ' 

7 Number of Courses, Questionnaires Administered, and ' 
Planned Input for Courses Visited . 15. 

8 ' Per Capita Cost Data from 1981 TAEG Incremental 

Costing Model.. v ' ' 22 ' 

9 Percent Structured Interview Responses 'of 

Three VARII by QI Variables..../.... 2 4 

D-l Mean Values and Significance- of Three Variations of II ' 
Across 50 QI Questions.. ............ 62 



/ 

3 



Technical Report, 147 



SECTION I ' * 

INTRODUCTION 

. Sel^-paced instruction in the Navy is being criticized for producing 
graduates who . have- not retained the information taught, and who cannot 
perform on the job. 1 IrKjfhe face of . such criticism, some self-paced courses 
may be converted back to conventional group-paced instruction, . 

*»• * - 

Unfortunately, there is widespread misconception in the Navy about 
self-paced instruction. It supposes that the huge differences in student 
abilities * can somehow be accommodated by merely allowing the time for 
learning to vary. It also supposes that when you call a course "self-paced", 
"it is. equivalent to ' individualized instruction (II) . In^ fact, 
individualized instruction requires the use of a : . number of sound 
instructional elements; self-pacing is pnly_one of these elements; 

Self-paced' instruction,, with some- additional individualization of 
instruction, was implemented in the Navy to provide effective instruction at 
lower cost; This eriddavor has lead' to. a proliferation of yearly 200 such 
courses affecting nearly 100,000 students yearly. These attempts to modify 
strategy have yield&d a number of ways in which courses are taught. Some of 
these attempts have been, highly successful; others are criticized is 
ineffective. Disagreement" over ^the effectiveness, of a given method./ of 
delivery of 'instruction' ntay be due to the various meanings attributed? to 
.'Jstelf -paced" -ijis,tructiorf. '+* \. , 

f ■ It*'' *' 

Some "time^ ago*,* The Training Analysis and , Evaluation 'Group (TAEG) , 
poinfed\ oat VtfeiS; 'confusion ■ in terminology and recommended a comprehensive- 
.survey *t<y establish the types and extent of II in use >in # /the Na&y 
(Zajkowski, Heidt : , Corey, Mew, and Micheli, 1979). Later, Hall and Freda 1 
(1982) suggested that there are apparently many meanings to II, aft%that it 
is not a unitary concept. Both reports show there is a need to identify and 
to differentiate 'the variety of instructional practices now categorized as* 
II in the Nav^l Education and Training Command (NAVEDTRACOM). Consequently, 
the Chief of Naval Education and Training (CNET)' -tasked, the JAEG to conduct 
a study of the variations in this type of instruction. 2 . ' 

PURPOSE OF THE STUDY . ^ 

The- purpose of the present study is td identify -and document the range 
.of instructional activities present in courses thought ". to* be 
'"individualized" by the NAVEDTRACOM. A model of individualized instruction 
is developed to provided basis for qualitative assessment and as a guide 
for development of courses. Proposed^fchanges to existing directives are 
provided to guide course jdesigrters in creating Appropriate-, efficients and 
effective instruction. ' 

» . > . * - ■ 

Commander Patrol .Wings Pacific ltr ser 70/1065 of 31 August 1982. ' 
2CNET. Code, 022 ltr to.CNTT of 5 April 1982. 
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DEFINITIONS 



Three categories of instruction characterize the treatment that will 
follow. They are:, conventional, mixed, and self-paced instruction. 
Individualized instruction can occur witR in these categories- by degree's. r 

CONVENTIONAL INSTRUCTION (CI). The central features of CI include group- 
pacing, lectures; students selected with similar' academic, aptitudes, and a 
single form of instructional material'. Conventional instruction appears to 
prohibit features that would individualize the. instruction; however* many 
instructors ■ have developed subtle; ways of individualizing . within the, 
constraints, of this seemingly invariant models 



MIXED INSTRUCTION (MIX). 'The term "mixed instruction 1 ^ is operationally 
defined' to represent, courses reporting betwefen 5 and 90 .percent- "self - 
pacing" in the preliminary survey conducted, for ^this study. " 

SELF-PACED INSTRUCTION (SP). This type of instruction allows student*; -of 
different aptitudes and previous knowledge to progress through a program at 
their own rates. While not: Inherent in the definition, many, as same- the 'term 
"self-paced" ^instruction -ipcludes many >of ^tjie/ other characteristics of 
individualized instruction in addition to. releasing the student from time 
/constraints. This Study does not make such assumptions. *• 

INDIVIDUALIZED INSTRUCTION (II) This term is defined as instructional^ 
activity designed to accommodate individual human differences- 1 in background, 
skill level, aptitudes, and Jearning sty^s^haracterize'd by (1-)-. releasing' 
of tiirre constraints, (2) choice of instructional media, and (3) adjustment 
to skill levels and learner characteristics (Zajkowski , et all, 1979);. 

.Additionally, those operations in traditional classrooms, self-paced 
learning centers, an&^ mixed- environments that, bring about/ °the/ 

, individualizing of the instruction* are -identified./. These operations, are 
further def ined in. tlie following paragraphs. ' ' - 

QUALITY OF INSTRUCTION (QI). ' Six -elements . of quality instruction' will ! be 
used in * studying all of the courses selected in> the/ sample used in the: 
present ^study. - They are' based on the work of, Bloom ^(1976) and ; his , 
colleagues wtfo . have developed 'a system of Learning for Mastery (Bloom, 
1968^ Heine,, courses are assessed in. terms of the extent to which the 
^bilov/ing six elements (identified by ( Bloom) are present: 4 



\ 



1. Prerequisites (PRQ) . These are the cognitive entry behaviors that 
are measured by the Armed Services Vocational Aptitude ^Battery readingVand 
computational scores, or similar achievement tests*. Other achievement tests 
show the students 1 readiness for learning a particular lesson or module of 
instruciion. Affective entry behaviors are attitudes reflected in measures 
of motivation and perseverance*. The most adaptive' instruction accommodates 
student variation in both the cognitive and affective entry behaviors in 
deciding specific instruction for a given student.- ^ 



9 
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• Xues (CUE): Thes6 are the^ signals telling the students what, it is 
% ' they ; must :, learn. They /aire a major part of the instructional materialsT. 
\* package, as S^ll . v.as* thV oh-the-block instructor's- lecture,. Learning : 
.objectives, the format of printed materials, the rubrics and headings, ( 
graphics * diagrams, mnemonics, 'demonstrations, topic sentences , and- verbal 
prompts are but some 'of the cues tl^at bombard . a student in the typical 
•cVassrooriri^ learning 1 center, or laboratory. 

3. * Participation (PAR) . This is the extent to which students are 
. V . given opportunities to practice and rehearse that which they are to learn. 

It" includes exercise materials 1 immediately following a small segment 'of* 
^activity 'and: the distributed practice of thjs activity over a period of 
v tiineV To" learn something, the student, must do something. .This "doing" may 
\ be' either in the form, of drill , exercises, or quietly, thinking about 
' -y : - something related to the* module "of instruction.. Since there is a very- high 
.relationship between intensity and time* spent with" amount of learning on a 
• k .:• tppic, -it is the aim of effective instruction to keep the student's mind 
V* engaged in the subject- , matter.- Daydreaming cannot . count as study time. ' 

Appropriate teaching . techniques and appropi ate practice materials caaa ejicit 
''. y • the high degree of involvement necessary in students to hasten them along 
pat^s of learning.. Breaks in the claSs period can. be productive; in PAR only 
if the students talk about that which they have been doing. { • 

4. Reinforcement (RNF) . ■ Reinforcers strengthen the behavior that 
precedes them. / They should be given after appropriate behavior and withheld 

• after inappropriate v behavior. L< Reinforcers are. idiosyncratic to students;* 
" however, there are some reinforcers that are generally applicable. These 
include^ praise, recognition, special privilege, and, indirectly, feeliirgs of 
achievement. • ' _ •* 

. J ■ . 5. ? Feedback (FBK). ( Providing - students, with /information about 
performance serves not , only to reinforce successful 0 behavior, bgt it also 
provides guidance on what to study and how much effort 7 to* expend in meeting 
course goals/ Tests, quizzes, self tests, critiques, oral and comprehensive 
examinations . al l provide the student information , as to ,how, t|iey are* doing^ 
with, relation- to enabling and^ terminal l%arning^^ectiveV. Such 'activities 
are an important part in the" design of instructional materials. 

6T Correctives (COR) . After feedback/shows the student that there is/ 
a difference between the ; demonstrated and the required performance ;or 
practice, adaptive' instruction calls Tor a prescription to get the student 
back on tr?ack. Correctives are* -'.the pres l cr i bed alternate forms, of presenting 
that which the*student is to learn. The summary, narrative, ■ and programed, 
instruction mentioned, in NAVEDTRA 110A qould be used as correctives in 
certain situations. -Correctives are the learning activities that, adaptive 
instruction uses to ensure that all students have repeated opportunities to - ; 
'^arn. v " * % -. . 
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PLAN OF THE STUDY . • \ ' 

The study was conducted in the following sequence of activities: 

4 conducted^- a review, of ^ the literature* concerning j II - (presented in 
appendix EV • ; \- V- s - ' ' } ; . ' 

• . distributed a. Course: Description survey to -a population of- 201 

courses categorized as individual iz-ed. in struct i on * ■ 

. established categories "for assessing \. the- • variations in 
instructional practices,. : : . . , 

^ • determined, a sample' of /.courses... for site, visits and detailed*' 
analysis;, of instructional practices and materials used .* 

, • site visited 37v courses and administered : a structured^ interview 
.-. and quality of. instruction questionnaire to a sample of students, 
. /instructors, and supervisors ; 1 . 

y categorized courses as; conventional, mixed, or self-paced" 
instruction and examined the degree to which elements of quality 

* . instruction in each category were present'.' 

ORGAWZATION OF THE REPORT =* \ . - ° : ' 

In addition 'to -.this •in1:if:dduction v '.the-' :: report xo.ntairis- ! jFour sections and' 
six appendices. Section. II provides a detai led description of 'the approach 
taken in the study, arid . also describes the 'Sjamp'le'.ahd ./instrument; us.e'd .tov 
.gather data. Section III;, presents, the., results of the' anatysesXof data. 
Sectiorv IV is : a 'discussion i of "the results. Section. V provides conclusions ; 
and recommendations. Appendix A contains the Course Description Form with a • 
tabulation ofVesponses .to. each.,, item-. ■' Appendix B' presents the Structured 
interview Form with- a summary of course- data organised by 'three -categories 
of II. Appendix C contains the Qua.lity of Instruction Questionnaire and the^ 
scaled values for each item. Appendix D presents the mean va-llies for each 
item ofc the Qual ity of 'Instruction Questionnaire as^ a functibn of type • 'df 
II. Appendix, E gives* a review of .literature pertinent to the .rationale for 
using the Quality of Instruction variables in the . study. ' .Appendix F 
synthesizes the literature and study findings with a ' model of ideal 
individualized instruction. " , * 
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••. V"* ."' ■ ; ' ■ ,K ' \V;W ' . ■ : " Vv ''; "V 
:V » : \V' ,: v ^^"y SECTION- r \v\ : "^V/v ; ^f; 

.v '• ^ . TEGHNiCAL APPROACH Vy.'"' ^/^v ■ 

G, This - section describes th& sample, vthe^ .'develppnient . of . th&:takr$e mm . 
. .Description Form used to obtain* information descr^bT^ pf .:'V : 

. II ^dogrses, ,*;tfie ... procedure for decid^^ to: visit fo'r;^#.fe..:\ ."i 

y ■"■ v detaTled' information, the development *. Of; :a i. quality of, instruct iony. 

questionnaire used to" assess instructional;: pracCTca^^lthfn.. the variations #r ,/*'•: 
*6f Ili an analysis of . .itudgrit time taken to; mas tef ;.i ns truct ion , '.. a .structured 
"■■ * i n tervi ew-^'proCedu^e ajid^ ^he- rnaaor ' data:- analysis ^rocedQrgs> v'Thwaghqut- : .y/J- 
this section, there 1 -* are ^triptions «f sal iertt; characteristics of courses^/^ / ! 
falling in . each of three categori es , *orf variations of .individual 2^id : :: " 
- : \ * instruction- (VARi^): conventional ; mixed, and-' self Spaced* " :.'./ ■:> \' ; 
: ■' ;A." >v~' ' .. .. ' . •• .- .'r-\^ v •.'/"',>"■ V" " 

, DESCRIPTION 0F V THE SAMPLE - - • a • : > r . - v \y& - ; V 

- .; : The -study began with a -.sur"^^*6f;;;cburses in "the -FY 1980 Master Cqu| 
Reference File 'of the^avy- Jnt6 : grated Training and-. Resources Admin f 
System/ .(NITRAS) : which "were listed as sqlf -paced, .conlputer-mahag^ , "gr -\ % f 

1 • having a combination of methods of inHiviciualizing' Instruction. '., T%:js^seahShy x ^ 
yielded 199 Course Data Processing] 

in'struQtor-managed .instruction, 11 ware pomput er-managed ihst?uetibh-, "and •* : 
119 were a combination of these- two methods c Thesfe ;^ 
, ^ NITRAS: as "P," [ ".C, ,, -.or VB? 1 respedtivel> 




To -seYect- the. Sample* of courses for 'more. detaiTed*-anal^sis, a- Course 
Description Form 4 (CDF) was designed toi 'el fci t . i^fomatidnr concerning 
instructional . practices.- being " Cised in courses identified. , in- ^ITg^S as^ 
containing some degree of yn>- The questions generally, dealt with; time,' 
administration, and phtldsophies: . . • ^ /+ 

; >r/ - •• ' ■ • . . v . 

> 1.. * The use of" instracter; §tiMent^. study, and awaiting instruction 
. : ^ ' ^ ' timers- L Specific, questions ; deal : with ; ^ use, . recording, and , 
^ percepfij^ of thes 

, 2. Course . administration 3 and organization: ■ TTiis category/ of . questions 
includes sucJ*^concepts/as criterion for. mastery, use of feedback, 
. t^pes ;of* reinforcement employed, media and.^lnstructjpnal^^td^ial*,' 
used, 'numbers and type of personnel assigned, an£ 'relative 
pro^iprtions^ of instructional type (lecture, lab, self-pacing). . 

3. rf Training philosophies: " This group^f: questions: assessed ^beliefs 

' j. and attitudes about -,the ^s4c. .;^ M earn ? -the 

level of achievement;^ of students, and 

the pontfrol^ ' 

; See 'appendix^ A for specific questions dealing with- each, of these areas. •# j- . 

• CDFs were mailed „ to. 83 of the 199 potential targW. courses^ . 
CDF was mail ej^ to' courses having identical coarse identific 

•.*''' ' . •. ■ . , ' ■* " ^" • '' ' 
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titles, but- varying CDPs at the same location.'. A^ exampi/^ir the- Basic 
>. . Electricity and' £le 
^OrlandOi; 30 at Great Lakes, : -27 , t-at*-.'§ah : •irfo&o,'-. and-- '9----afc > -4teni>ftfs-;v. •.' Each- 
. location received only one. CDF. • Table. 1. summaries' the number of courses' at: 
" each location and the. nunfber- of. CDFs'sent an8 /usW . .i'rt : this study: IncludedV 1'' 
•; in table 1 are two courses that later became available f^thg^lW'XorpV*< > 

Communication-Electronics School (MCCES) . * They were:- '. a computer^assisted ' 
,. ./and .an instructor-managed ..course from MCCE3, ■Twentyrii.he f Palms, 1 California. . ' 

Both were versions .of the' Communications Center : Operator k 's -Course (CC'OC) 
.. taught by differing methods.- Of the 83 CDFs' sent, 78" were returned' or . • 
accounte d f or— a- 94 percent return ; rate". Z Ba sed on these ±e£nrhs2 ft4 ™ M rses 
wei*e. suitable for analysis. 



TABLE l;. POPULATION OF •'nJTRAS COURSES- BY LOCATION ' V-?'V- C J: . 



Location 



LL 



Number of P, " 'Number of Course ' Number 'of CDFs 
C, ,or B; % , DeseriptfoV* ' Tnc 1 udecf \> 
Courses . -^Y^fm'Seint ^^ in Study. 



* SSC Orlando, FL ' 

** NTTC Xorry- Station., FL 
NATTC Memphis, TN 

* SSQ, Great Lakes , IL y 

* SSC San Diego, CA / 
FTC San, Diego, CA • 

* NTTC Mertdian, MS - 

* FTC Charleston, SC i 
*STC Charleston, SC 

NTTC Treasure Island, CA 
*• FTC Philadelphia, PA 
NSS Groton, CT - 
NATTC Lakehurst, NJ ' 
FTC NorfolV VA 
NAMT MillingW,- TN- 
STC..PAC P6arl Harbor, -HI' 
**MC:CES 29 Palms , CA ' 

Total " : 7\ - " 



■Jr. 



.27 

' v 14 

35 

' -48 

^ 3 7 

" -5 
.3 
2 

: . ■ 2- 
• 4 
'3' 
9 

.2 

- 1 
3 

. -2 



201 



S'2 
U3 

6 V 
' 16- 

-.8' 
• 7 

V 

' - 5 

i 

■ ■ 2 

; 1 

1 6 

2 '- 

i 

3 

. 2 
§5 



J? 

:6 „ 
11- 

; 5 
•i "" 

5 

2- 
2 • 
2 • 
4 

.3 • 
6 - 

■ 0. ':■ 
\i'- ■ 

.0 

• 2 



-64 - 



* Locat ion s visited. . ' 
**Sampled course not in NITRAS. 



ERIC 



f v 



V . • x 
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Tab)e^2^ists^the 64?coursq£ from; which- usable CDFs' were obtained. • The' 
table ^giyes a\* 'sequence •/number, depicting the order the completed*.' 
.questionnaires^ CDP numbers, course short, title,' locatibnY' 

tlie "estfjjiate'd .ipercent of the, course that was "self-paced" (Question "No. 34" 
.on, the CDF) , .and the- number of- - stud^ti in .annual' planned ihput v jfa 
atfditioni the- courses receiving a site visit are indicated by an asterisk. ' 



^ The CDF was sent to the person "most knowledgeable" of 
, a course.-,- A' cluster, analysis of -irhe 'responses' to the' original 
' .i^id inot; •reveal clear patterns associated with variations *. in 
lthis-wi nf ormati on— was— used— tov^'e;]-e'GtM:he^wi-des^yar-i-.et-y— of-^- 



the selected 
62 Navy CDFs , ' 
II; however, 

_jULlth-i^infomation— was— u — - 
V vpna;ct}ces; ^n courses at a given; training location. CDF 'question No: 31 „' 
> "What percerrt' of" this course is 'self-paced '? " ' had the hi ghest rel ationship- "" 
.",'* Wi th. '; v ari at ion S'lin' I I ( VARLI) . Thus., this question Became the basis for, the : 
/ ; .; .^categories -of. Moused .in .this-- study. • The , percent of self-pacing in each of. • 
-the .64 course^ from which-, CDFs were" used iY listed /in table. 2. i- . 

DESCRIPTION OF THE CATEGORIES OF II r 

•. '.<:'■ <■ '. ... * '■' v • ' ■ '■■ '. t ' ■• 

«Three. VARH were identified. t There we t re 7, 10, , and ' 20 courses, '•' 
respectively, -in the three categories. Those courses reporting 0-4- percent 
,self -paced on CDF- question 31 were ■ classified \as conventiopaV instruction 
(CI). Those courses 5 tov 90 percent self-paced were classified mixed (MIX] I . 
- Courses- with 91- percent or. greater . were self-paced (SP). The' analyses 
.described here and the .results reported, in section III are based on these 
categories.; '« «* • 

; . Additional data from the CDP 'were analyzed to provide a more" detai led • 
' -description of courses in the •three. VARII. • Tabulations were made 'for the - 
• CDF questions eliciting frequency "data; ;arid univariate' ana.iyses_of_ya'riance__ 
. were utilized .for the interval- and- -ratio-seal e'd- data. The -following: 
• describes - some, .additional characteristics of ' the three categories Of 
courses:' >The most i nd i y i d ira'l i zed ■ courses appear" to' be more likely - to/ let 
•~ the student' determine when- breaks were -to occur during the class' day. In ^ 
''addition,- thesef courses, show more variability in the -time' taken to' finish 
..the course. The three VARII differed in the hours -in a typical class, day, 
the learning minutes in a.- typical class -hour,' the criterion ,f or mastery on a ' 
.lesson or module, the amount of lecfuring, the" amount of self>stu'dy, and the 
use of study' materials. Tables,3; 4, -and. 5 provide summary data for these 
findings. / ■ ' . - • •< 



: H#-'.. 



9 

ERIC 



li 
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Technical Report' 147- 



TABLE 2'. COURSE .POPULATION 



Seq CDP(s-) 
No. "Y- 



Short ' 
Title ^ 



Loca- " - %SP. .Planned' 

tiQn ' (0>1) > Input. 



1. 3665-3698 (3)'' 

* 2: 604H-6550 (24) 
*'3.. 303N . 

* 4. 4 .3197 

* 5.' 6458 . s : 

6. 417M 7 

7. 6059 . 
*8. 6522 

* 9.-6102 
10. 6057 . 

*11. 16061' ' 
*1 2. 65.01 
13. '6161 
*14. 6320 

15. '. 6302 " 

16. 4376 ' ,. •• • 
17*. 6319 .. 

*18. 6380 : 
*19. 6381 . 

20. 9331 

21. 281 V ■ ; ; > : '■ 
*22. 403V 
*23.' 5382 . • 
*24." 601R-6551 .(30) 
*25.- 6144 . ■ ' * 
*26. 5152 .'•' 
*27. -5202 -:■ 
*28. 6269-6549 (27) 

29. 6119 . , - 

30. 2589 • '"• , ' - 

31. 6301. 
32- 6020- v ■* 

*33. 8511 . 
. 34,.' 5340 
*35. 6339; 
*36; 3218/ • •.. • 
37. 2859 
38'. 6492' 
39. 6493 
*40T 6486 \ . 
*41. 6488 '• 
M2. 6261-8562 (.4) 
M3. 648-7 / 
44. 6280-6284 (4) 



TTA ' ' Orl ando 

; . BE/E/. - - . Orlando 

.* CTO TACSOM " - Corry 

•CTT ELINT OP Corry 
• -CTT-WBS OP — — 

NSG ,;Jdoc ; '■■ 

. SK A *. " 
, AK A 

• PN A .' ; 
.-• YN A 

DK A '. 
, AD Al , . . * 
;. CTM A 

■ CTT SPE 
. . CTT A Prep 

CTT SNMC/O 
' - CTT/ICR/Flex' 

RM A Se.a 
.'. RM A Shore 
• ; " PCQ/PXO Rev .,'. 
' 5 Corosion Cont 
LC Instr .. > 
• MSE CODE OP. 
BE/E 1 ■' 

'Wf A Basic, . * 
BAS DIG Fund ... 
. •■ GPETE Bas Q'p V 
'..•'BE/E .-V: 
, ', *: HT A-1V" 1 .-' 
FUEL PROBE- - 

■ ' CTR A ':' '"■ 
V^CTAA 

~S " '.J GAS FREE ENG 
V DC REP PTY "LDR 
HT A-2 - > 
",..•* - DC ASST * . v.'. - 
. •. - ""BECTEC ' > 

MMCLA1200" « • Great 
• : - ? MMCEA 600 , Great 
BT CL A 1200 V Great 
BT ADV "OPER: • Great 
PE Great 
.EN fit A Great 
AV FUN Lakehurst 



Corry 
Meridian. 
Meridian ' ... 
■" Meridiari" . 
Meridian 
Meridian 
Memphis- 
Corry 
Corry 
Corry, 
' Corry 
Corry . 
Sah Diego 
San Diego 
San D-iego 
^Millington 
' Great Lakes 
.San Diego - 
Great Lakes ■ 
San Diego 
Charleston . 
Charleston 
San Diego 1 
Treasure IS 
- Treasure IS 
Corry 
Corry " 
Philadelphia 
Philadelphia 
Philadelphia.: 
Philadelphia 
Great Lakes , 
Lakes 
lakes 
Lakes ' 
Lakes •• 
Lakes 
Lakes 



4s» 



100 „ 
100 
0 

75 
90 

: 0 

100 .• 
90, 
100'- 
100 
100 
60 - 
5 
■ 5 
,0 
100 
. 0 
" 0 
. 0 
.30 
100 ' 
100 
.100 
100 
100 
100 • 
100 
,100* • 
0 . 



S?k' '0 



0 

.. o- • 

. 0 
33*. 
99 

; so - 

:/,0 y.: 

■ 85 ' 
.90; 
90 
0 

10O 
65 

••'85- 



. 64 
" 3Z08. 
; 140 
'.'100 
—100- 
70 
1079 
. 722 

■ 1263 

• 1844 

350 
1832 
405 
700 
1325 
■i 50 

• ioo' 

,1582 

• 1545 

15 
***, 

'350* 
400 
4761 
. 3677 
" 295 
v 375^ 

• 5654 - y 
1124. 

■ 85 
' 615 
. ' 240 

75, 

•83 
1086, 

198 

304 
2250 
2.737 
2250 

600 
10248: 
11361' 
1605 
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f ABLE' 2 NcDURS E POP ULATION ( cdnti nued ) 



5eq - CDP(s) 
io. 



"Short 
Title 



Loca- 
tion 4 



%SP 
(031) 



Planned 
Input* 



45. 6519 : 

46. " 4509 

47. 7764 

48. 7:7j65 

49. ^7766 
1*50. 602D 

51. 2694-6046 (3) 
*52'. 8981 
*53.* 540 J 
*54. 604J-6243 (6) 
*55. 601B-6237 (9) 
*56. 6210-6229 (17) 
57. 6521 
1*58. 402P ; ^ - 

59. 501X 

60. 022B 

61. -501C 
*62. 6047 
*63. USMC 
*64. USMC .* 



. PR Bas 
PR Adv 
NP/I/ 
NP/II/ 

-NP-/I-M/ 

•EW A 
IM A 

TMA/SSPP- 
ClAC 

AV A • . 
BE/E 

afun er 

TD Al 1 
LC INSTR 
GPETE Bas Op 
DIESEL Od 
OX ANAL C CMB 
OM A 

CCOC, (CAI)- 
CCOC (IMI) 



Lakehurst 
Lakehurst 
Lakehurst 
'Lakehurst 
-Lakehurst- 



Corry 1 

Great Lakes 

Charles ton 
.'. Charleston- 

Memphi s 

Memphis r° 

Memphis 

Memphis 

Memphis 

Groton 
. Groton 
MA Groton 

Great Lakes V 

29 'Palms 

29 Palms ' . 



0 
0 
0 
0 

— .0- 
87 
99 
50 
0 

100 
100 
100 
,100 ■ 
100 
100 
0 

■100 
100 
100 
100 



' . 575 
** 102 
. 139 
2 -. 
-2— 
912 
• 79 
: 240 
460 
4600 
7855 
16860 ; 
361 
195 J 
392 

218 ; 

16 

57 
214** 
434** 



* Course visited. " 
** FY '82 throughout.. ' 
***Course ended v 4 Sept .80. 
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Ti\BLE 3. f^AN. CLASS TIME 'AND CRITERION TO MASTERY FOR 
THREE. VARII (CDF- NOS, 3, 4, 14) *. ,'. 



V ACTIVITY 


V ; ' '■■ . CI 


\ Mil ... 




'.; sp . 


Class Da^/Hoiirs 


.." 7.71 • *'•• .,. * 


•'. ' • V *7; 40 * 






Class >ioj*r/Min. 


48.14' . 


/ Vo..'80 


4 

■ 1 


; 54*15 


Criterion -to 
Mastery 


/ • 73.00 


75.40 

. ■ \ • ■ 




93.05 


(The three. VARII differed significantly on all activities.) • - 

1 ' \ 




' ■* \ • 

TABLE 4. 4 PERCENT ACADEMIC DAY SPENT IN THREE VARI 
>• . CENTER ACTIVITIE-S (CDF. NO. 23)* 


' 1 \ 

L BY, SEVEN LEARNING . 

■■'\* ■ ■■: 


ACTIVITY 


• CI :\./^-;, 


MIX 


rX W- 


' SP " 


Lecture 

Discussion 

Demonstration- -* 

Self-Study 

Tutoring 

Film 

Laboratories 
Other V" 


54 

8 " . 

4. . 

7 

• ■ , o / 

3 

24 ■ 
0 


11 

4'' • 
' \ •* 5 
53 

:.• .. 2 :. 

.3 . 
22 

1 . 


..' ' .V ■■ 

' . \ - . - 


£ 

• 3 • 
3 

53 

3'/ •' 
T 

32 • 

• , 3 '..< 


*Does not add to 100 due to rounding. . 








' TABLE 5. 

p 


' » '*■}'■„ 

PERCENT CLASSROOM- SELF-STUDY TIME IN THREE LEVELS 
INDIVIDUALIZATION BY SIX TYPES OF STUDY MATERIALS 
(*CDF NO. 25) . . ' _ . 


OF 


ACTIVITY ■ i ' J ' 


CI 


.mix/ . 


* 


SP • 


Student 'guides 
Sum, Nar, P.I. 
Other P.I. 
Handouts v 
Equipment Manual 
Other . 


16 • 
0 
,0 
48 

14 < . 


9 - 

. 56 ... 
0 

15 

,1' '■ 
9 


0 


17 

51 V 
■ 4 
13 

1 -7 , - 
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• An important difference in the variations % 6fTlI is in the - ratio of* 
students to instructors a'nd" supervisors 1 " (.table 6). * The .SP learning centers 
have over five times as many students,, on the average, as do the CI. classes, 
while there - are only- about' twice, as' many instructors, "present. ; The 
.student/instructor ratio f(S'/l) for SP is. abdut 18' to i, while in the . CI 
environment- it is about 8 to 1: • '' *' 

■ .:• . / . • . ■ ■ ... ' 

TABLE 6. - MEAN ^NUMBER 0F V PERSONNEL IN .CLASSROOM OR LEARNING CENTER ; * 



7ARII— N SDNTS * AIDES ' I NST \S"UP ERV S/I 



{CI '7 8.86 0.14 1.14 ■ 1.&6 7.77 

. MIX 10 19.50 0.30 ~2.00 2.60 9:75' 

SP 20 45.30 , 0.60 - 2.50 3.70 .18. U 

Total .37 ' 31.43 . * 0.46 2.11 3.05 . 14.90 

— — » — 1 — ..I I • 



Of. the -64 courses summarized in table 6, 16 (25 percent)*had S/I ratios 
of six or less. These courses "break down to nine CI (56 percent), three^lIX 
(19 percent), and four SP (.25 percent). i 

QUALITY OF INSTRUCTION QUESTIONNAIRE ' ' . v , 1 

A quality of instruction (QI) questionnaire was developed to assess the 
extent 'to which prerequisites (PRO), cues (CUE), participation- (PAR), 
reinforcement (RNF-), feedback] (FBK),' and correctives (COR), are 'present in 
various types of individucyiized ..instruction.. - Items were constructed 
according to Bloonfs (1976) theories of ^Learning for Mastery and were 
adapted to. military training situations. Each&tem was examined for content"^ 
and structure -by TAEG ■* staff members. -The interim QI was field tested with • 
students from, the Orlando BE/E course. <n addition^ instructors and 
supervisors'* were asked to review the questions; The final vfirsfon of the'-QJ 
used in the study was based on suggestions from these" students, instructors, 
and supervisors. *Fhe resulting questionnaire consisted of 50 item? that 
could be* administered in approximately 20 minutes. Appendix C presents this ' 
questionnaire with the obtained scale value for each of the variations of II 
on each item. *. , -—^ 



The ^1. questionnaire was administered vto l-,090 ' students,. 170 
instructors, and 54 supervisors involved with 37 courses at 9 training' 
sites. The combined igroups yielded 'an alpha reliability coefficient of .86' 
on the instrument. The scoring templates were scaled 1 to 9 - f or data 
analysis' with the polarity reversed on appropriate scales; so that results 
could be consistently interpreted. The questions were grouped according to' 
the six QI categories (Bloom, 1976) to facilitate the examination of the 
impact .of these variables: 



I 




* • - . \ . ". ... • • ■ 
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Prerequisites (Q1-Q5, Q7, *Q25) ' . ".- 

• "Cues (Q8-Q14) , • ' *" , * ■ ; . 
Participation (Q16, Q18-Q20, Q22, Q23, Q26-Q30). ...... . 

'Reinforcement (031-Q38, Q17, Q21) • . ./" r .. " 

v Feedback? (Q40-Q44) • . . . 

• Correctives (039, 045-047, 949, 050)..- . , .v .. . 

/ Later analyses of questions- 6,. 15 i 24; arid' 48 found .them not as related 
tb the six QI variables as anticipated, although their results are of use;to 

the st u dy. — - ■ ; - - — ,. - . — — ^ — L — , — . 

Table . 7 shows- the distribution of QI questionnaires ^- ^6 "courses, in 
each category of II and the yearly planned, student input for each of these, 
.categories. Data obtained from'. these questionnaires were submitted to 
detailed analyses . of variance of.VARII and type of respondent by each of the 
50 questions and questions grouped according to the six elements of QL., 

' ' *■ •• i ' .'■•".•'*' 

• TABLE 7. NUMBER OF COURSES, QUESTIONNAIRES- ADMINISTERED, AND, 
J. PLANNED INPUT FOR COURSES ' VISITED ( 



VARII 


(N COURSES) 

f 


QI . ' - 
(N RESPONDENTS) ' ' 


■ -PLANNED , 

'- INPUT;';' , 


■ % PLANNED 
INPUT 


- CI 


' • . 7 •• 


• 

• '.146 v ■ , ; 


4,472 ' 


6 


. MIX 


• 10 


210' 


- 8,915 


12V 


SP 


* 20 • 


-95 ; 8 


62,489 


■ •. 82 


' TOTAL 


37 . 


. U314 ; 

• '•. ' ■■■3' "» ■ 


-75, '876 • 


, 100 ■' 



TIME TO MASTERY ANALYSIS • • 

The literature of Learning for:Mastery predicts that the time required 
for ^earning in a diverse group of Students will become more homogeneous the 
longer they, participate in efficient 'instruction (Bloom, 1976; Anderson, 
1976).. This is contrary to^e^traditional notion that student achievement 
'.becomes more- 'heterodeneous^^Monger they stay in school. One aspect of 
the present study v^as to "*"fe^w??f or ' an index of ' such homogeneity in time 
required for learning ' as ■^■.'^a^ure > of instructional efficiency. The 
coefficient of. variability (v - ; s/m) /was chosen as the index; where, v. = 
variability, ;s = standard deviation Of student time,, to mastery on a given 
moduje, and m. - mean student time to. mastery on the module." If students 
requiring longer time -for learning begin an individualized course of 
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study^and become piore>d more like the faster \leWer S ;vtheF' the '-stahdard 

nZnil n ,^ / eC 4. re i aSe tu r ' a 9lvw cohort as- they progress 'from module to 
module. Unfortunately, these modules are not of the same ., length or 
difficulty, so this makes it possible for the variability. : Q f the lonqer 
ISSJf! J^ re ^ se / .The coefficient oP-\var^lt41y.:^dt.1!d^iih^^ch- 
uh >h This procedure was . applied' to a selected- sample of courses "for 
which there were time to mastery raw data available* . • ' : /■ 



STRUCTURED INTERVIEW 



. • •• In , addition. to^tne ; quafe of instruction questionnaire,- a structured- 
P^onifnf 5 ^SW*: io" specifically assess the degree to- which" Bloom's 
elements of quality instruction were present in the instruct Tonarmaterials. 

;. Consequent y, the ^tured interview had six major sections, each dealing 
Wltl ?. . an , element of- =01 (see appendix' B). The structured interviews - were 
administered to the,, person • deemed "most knowledgeable" of the course by onen 
of the two. principal investigators,. Analyses of/.the responses are based. on 

, a total of 37 interviews, one for" each of the courses sampled. . " 

_ > In conjunction with these* interviews., , instruct iorral SmateriaTs for each 
■2i-£2*S 0U £ se * were RhysicaHy examined and .assessed for the "degree to which 
l ? 6 ^? QI -were; present. Finally, samples- of reading materials were 

tahn 1 i ¥h ^/adt't' -*u ' J aU f * om the interviews were Analyzed by cross" 
tabulating the VARI I wi jth eaVh question and the six 01 groups of questions 

EVAbUATlON OF NAVEDTRA 110 ( SERIES) GUIDELINES ', ' 

A separate analysis was conducted of NAVEDTRA 110 (series) guidance 
with. respect to. Quality of Instruction elements.. For the purposes of this 

SSS 1 ^tS*i!! leS , tl - 0n -5 the structured interview was considered to be an 
aspect of the Q I -elements for which guidance could be prepared. Aspects of 

NAVhDTRA 110B, the proposed revision to NAVEDTRA llOA, which is the 

S^NAVE^R^mS* 1 ' 0 ^^ 1 " 9 I he .development ' of instructional materials in' : 
the NAVEDTRACOM. Based on the discrepancies observed, recommendations for 
developed'™ NAVEDTRA 110 (series^): instruction 'and far it ."use were • 
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0 
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* r*. I ■'• * .'. ■ , Technical Report 147 ~ 



t 



SECTION III... ■ • - : : ; 

' •■' ' •' :. '. • °*\ . : . ' • ,'- RESULTS; "'. • .:. v. i. ; ->:* : 

.. \ " '. ,. .-• . ■ *-.-;•. .'.,..< ; - f-,. ••• • ' , . ./ ', ■ ; ; 

s - •• This section sumrnar lies .the results - of the QI questionnaire analyses,' 

-.'.v ; time and .' costs : ahalysesy the^ examihatioh of course materials ■ with the 
-v structured 'interview, ahjJ/on-site: observations. ♦ 



QUflL ITV ' QF I^TR'UCTI^N . 



* . " /-Thje^neiw'T^bQ^'r^ Vespondeptsii % ' each VARII". on 45 of the 50 items On 
; . the QI- questi onnaire' was ; above average . . Th is indicates generally favorable; 
opinions toward ^almost' -every aspect of- instruction measured by this" 
questionnaire. The five unfavorable, exceptions (Q7;; 20, .23, 26, 45), cut 
v across categories of QI and deal, with highly specific aspects of instruction 
as opposed to generalized -trends; the most notable of these is the relative 
n ease with which students can daydream in self-paced instruction <Q23). The 
' most favorable responses showed that instructors were highly . regarded as 

•• ' subject -matter experts (Qll). and showed agreement in the usefulness of 
**.; practice activities for students (Q29). The reader is referred to appendix 
C for the mean scaled responses on. each vtem'Jfor the three VARII. ... 

The-QI items were combined Jsp .obtain information concerning the six 
• elements of quality ■ instruction,, as mentioned previously in this •section. 
,*" These six QI (PRQ, CUE, PAR/ RNF* • FBK, COR) became the dependent measures 
^\ 'for 3X3 ANOVAs. The' independent variables were 3. VARII (CI, MIX, SP) X 3 
respondent categories (student,' instructor, supervisor). . • 

Figure . lu shows- QI scores as a function of VARII* Analys-is . of the data • 
indicates CI to be judged by ; . respondents "as providing the highest .degrees of 
CUE, PAR, and RNF .relative to -MIX . and SP-r .MIX courses were rated higher 
•• than CI and SP in*?J<Q and COR. SP was estimated ^.-to .be superior to CI and 
MIX only in feedback. A more- detailed analysis of this, rn^in effect is 
. - i presented in* the following paragraphs. - 'Appendix, C gives a breakdown by. 
questionnaire- item and appendix D gives the mean values and ANOVA results 
.. : . ; for the VARII main "effect. : ' ■ ■■' '■ . :' '.*•' 

. • • . PREREQUISITES,. . The mean response to the seven questions comprising PRQ on 
, W- the qi s S hjwn • in figure 1, differed significantly, among the three VARII,. 
with the MjKa'rid SP groups- rated highest. This was probably due • to Hhe 
respondents An these courses finding the reading levels of student, materials 
easier (Q2) Jsigrcelving that more of the students are high in background for 
■*-■ '. success in the course (Q5), and. students being far more" likely .to progress 
through, the* course at a. rate of speed commensurate with stheir prerequisites 
for the- task (Q25). This is in spite of the .fact tha| the CI students have 
ter "attitudes' about school learning go ing; into their" courses (Q4) r : . . ,, 

, CUES. ( The seven CUE questions" combined to show, the CI ^ourses highest, as 

9 ' shown 'in figure !. '..Here, there was the perception among respondents, that 




ERJ 



jiiunn -in i i vjui u <L « ... ■ \~ 9 w . . v- ■ v- — r T r . • ■ j r . ■ . — ■ ._. ■ ■ 

the. learning jo^Je^ (Q9), there ; was far - more 

• instructor assistance in * holding the students 1 . attention *: to' the 
instructional materials (QlQ) , arid there 'was a greater proportion' of the CI 
instructors day being spent .in. giving cues to students (Q14-)V 
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PARTICIPATION, The Ml PAR quest ions taken ..together shewed, the CI. group 
\highest overall in this attribute of quality instruction /(figure. 1) . Jtere, 
the CI/ respondents : perceived students, to : spend significantly more t^rne 
beyond the scheduled gcademic work day* on their Courses (Q20), perceived the 
teaching methods in their courses to be more appropriate* (022) , felt'it was 
more difficult to* daydream /in c-las$: (023),; detected ..a greater degree of 
instructor participation in determining the rate of student- progress (026), 
saw more .time, spent practicing what students learn (Q27), and generally 
found more realistic practice;. act iVi ties (Q28. to Q30) . The SP courses*, were 
most^sk-i-l-lf u,Wi n-cut-t-i ng— the~t-ime— between— present 
and cutting the- time awaiting start of their courses (Q19) . . ' - 

REINFORCEMENT.; V, The third, and last' of the 01 variables on which the CI 
group' scored : highest, are the 10 RNF questions (figure ;1). In^ the CI 
courses, the instructors were perceived as more enthusiastic (Q31), and more 
likely to bel ieve * that all students can and will *, learn (Q33,. 034) . 
Graduates of the courses -were more likely to see the , importance of the 
course for their military career (Q37), and more likely tq recommend the CI 
courses to 'others (Q38). . The lesson materials were more 1 ikely to explain 
the importance of learning their contents ( f Ql7),' and were perceived to hold 
interest longer (Q2I). * . ' • 

FEEDBACK. The SP arid MIX courses scored higher than CI on 'the five combined^ 
FBK questions (figure 1) . These courses were perceived as allowing more 
tests and retakes of examinations (Q41 and Q42) , even though there: seems to 
,be less instructor involvement; fol lowing a test' (Q44). ■ 

CORRECTIVES. The combined responses to the six COR questions shown ; in 
figure 1, find the SP group lowest. This 'appeal an anomaly, ..since the 

theories .. providing the f oundat i ons for Mastery ' Learni ng - emphasize the 
importance of corrective activity. The. anomaly, is probably < due to the SP 
respondents 1 feeling that the time allowed for retearning aftet* a failed 
examination is too' short (Q45), that ther^ is less availability of:, 
instructors for helping students - (Q46), and the perception of less 
availability .af practice and restudy materials (Q49) , 

Figure 2 shows^ the relationships of the. three types of respohdeht 
(student, instructor, supervisor) and the six QI. var.i ab les\ summed' across; 
VARII. There were"\ignif leant, differences among respondents in four ;of the* 
six Q-I categories, ^he supervisors generaTly. perceived the greatest degree 
of r PRO; RNF, FBK, and; COR' present in their courses, the instructors were 
highest in the PARpresent, and "there were no ^significant differences among 
the groups oh the CUE and RNF. questions.. Finally, there were no signff leant 
interactions on any of ; the six dependent variables undergoing analyses of 
variance. 
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LEGEND: 

S0LID=CONVENTIQNAL INSTRUCTION 
SHADED 5 MIXED 

OUTLINED = SELF-PACED INSTRUCTION 



Figure 1. Three VAR I b^x QI 
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LEGEND: - ^ . v : • ' 

w SOLID = Students " ■ - ^ 
SHADED = Instructors ' . 
OUTLINE = Supervisors ^ 



Figure 2. Profiles of 1,083 Students, 164 Instructors, .; 
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TIME TO .MASTERY ANALYSIS . v / 

. Time to Mastery" (TTM) '■"■data were coTlected 1 from three sources,: (1$ all. 
Phase I 'Basic Electricity/Electronics students during; calendar year 1981. (N= 
v . 22704), (2j" a sample from Propulsion Engineering Basics (N s 62),:.rand. (3) a 
sample from Opti caiman A School- (#=19).,. These sources represent 1*4, 15, and. 
22 modules, respectively.- All data, were transformed to. coefficients of 
' variability. Figure 3. shows ja^-plot of these coefficients across the. number . 
of .modu1es''^^esehtjng. each course. . A' fourth s ource was take a from BE/E ;. 
• data presented in Federico and Landis (1979-) and Js also plotted /(FT) in : 
. '• .figure 3.. The general, trend of ; each . of the lines suggests the -decreasing 
variability of' TTM that* Bloom mentions,; but 'the rat.e of decrease in. 
.•toefficijent of variability is -unci ear. " - } ; 

COST ANALYSIS ^ . ' • \ ■;■.'■■'. .■." • ' V 

The Results of the foregoing analyses-prompted, collection of cost (data. 
» v This was already available : in, the TAEG Incremental Costing Model , And : ;a 
breakdown by three leve.ls : 'of . VARIT is shown* in table 8 (Dickinson and; Swppe,. 
''■;'.;' 1981)'. the three -costs ' compared here are": (1) Total cost per course' hour , 
v (2). Direct costs per bourse hour (not. fnc'l uding overhead), and (3) . Non-* 
' Student- direct costs per course hour (this subtracts. student salaries). In 
;''•"'. - all three; : comparisons the. SP costs were lower than the MIX- and CI groups. 
.This is -coincident with the changing supervisor ratio, reported in table 6; 
':'• '■" i.e. , f ewer .instructors eqi/aT lower costs; •r- -:>-'rv™-;vrr">'*^-" v • '•■ : r-'" :••:•-'- 

■V ; ' v , ' . ' TABLE 8. 'PER CAPITA jCOST DATA FROM 1981 TAEG ;V v 

; ' • "" "' INCREMENTAL COSTING MODEL •/ 1 ' \ 





. . . • k -■ \ 

VARII* ' 

■a-"'' " ; v V: 


«. .:',- Total Cost/. 

. Course Hour . , . 


'Direct Cost/ 
: Coursie Hour 

' * - 


, . Non-Student * . 

Direct Costs/ 
V Counse Hour 


J«. -. 


. ■ ■ CI . • 


. • .$111,79 *; 


$82.70 


? V $27. 06 '■ ; ■ 




* MIX • 

■ ■ ' - v 'r 


■W : • 96.64* 


'71,33' 




y 


• * SP y- 


0' • >v 64. 48- . '.'■;;■■ 


; : 47.66° 


1 .9.67 v 



7 The' direct- costs shown oh- table 8 are not as 'heavily, affected by 
* student: throughput as Vare r tdtal "costs,, .although.^ there, may be isorne v ' inf luence v 
■ > . of throughput on these cost§. "A ccihfjparison bf the 'average throughput, of the 
J *' ■ CDPs in 'the present study shows Tattle difference among the;CI, MIX, : ah.d SP 
, .categories. \ This rfiay. be due to some ^ of the ' tSrger courses in the study 
. ^ha^iig :> between 3 jaind 30 t CDPs ;ptr course ; '-1^t1jpn & ;($ee;iable- 2) ; The 
;^ muTtfpTe «CD?s per Targe, cdur^e ^nd tb^ bring tfe; throughput per CI, MIX, and . 
SP eatery toward equality^ - A further association of ^dlirses with these 
H ; ^,.;.cpst/J|Hiai.ngs was-' : bey6nd':the scope of :the present tasking.. , / - 



< ■ 




ngtf*3. Coefficient* of tori*** In Mofttfe Time to 



er!q 



23 ; 

-•.A. 



2$ 
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STRUCTURED INTERVIEW 



Table 9 contains a summary of responses to the structured interview 
v,data (appendix ,B),. showing the extent six - elements ~crf are present in 
^Iris^ 37 courses in the study. The data* 'in 

;the Jtflbli*^ of ' respondents indicating the instru^^nal 

-/element ; was 'preserlt in their .course.. Two types of values appear in tjable 9. 
• .The top • number ; in each eel 1 is. - the mean percent of the pertinent. 

questionnaire items answered "yes," and the lower numbers show- the range in 

percent of ""'yes*' scores for course. The Structured interview form and the. 

percent responses by VARII category are shown* in appendix B. 



TABLE 9. PERCENT STRUCTURED INTERVIEW RESPONSES OF 
THREE VARir. BY. QT VARIABLES . 



VARII'. 


• PRQ 


CONDITIONS OF QUALITY INSTRUCTION . 
CUE PAR • * RNF FBK . 


" COR " 


i 


CI Mean 
<■ - - 'Range 


;,. 20 

.0-50 ■ 

at 


.38 
14-62 


i . • 

38 ' 
0-88 : 


' V 36 • 
15-100 


42 

25-67 " 


, 52 • 
- 33-100 


1 

1 ■ ^ 


,MIXED'Me4rb 
. Range 


• 30 
- 6-50 


- 51 
24-86 ' 


' -71 ',*' 
1-3-100 . 


« * 

50 

.0-100 


58 •' 
. 42-83 - ■ 


■ ,70 
. 0-100 


[■ 

1 


"-SP "Mean 
Range 


48 

-13-75 

« < 


• 47 
' . 19-86 


63 

.■;0-ioo 


4.5 , 

o-ioo- ■ 


60 - 
33-92. 


.•• 63 
. 0-100: 





PREREQUISITES. - Observations made, wfii Te cbjlectiYig these data illustrate- how 
; ' Navy schools attempt, to- de|i1" with the prerequisite issue of assessing 
--skills. For instance, protests' are being used in 50. percent of the". visited 
self-paced courses to determine if entering students have the required math 
* ; 'or* typing skills. Pretesting is -a useful way to identify those who require 
■ r 9 special .-training before entering the main part of a course. ■ Some Scaurs e£ 
^ allow . students •', to "test o^t" .of sections of the course; without further. 

study,; if Students can convince /instructors they have* previously masiered 
•Vthe ski l|s" being taught'. However, ^students rarely take .advantage of -;thi£ 
. • ; offer. '.. ■■■■■■ """ ; . . ■ . • - 

*V - « It was learned that' as^ much as 50 percent of the content of lessons is 
. : . 1 isolated i nformati on nqt applied in follow-on lessons. Comments from course 
v specialists frequently included statements such as^ "...the lessons could .Be 
" taught in any "of der, ■ or, "...the .content of a lesson is needed in a follow- 
■ J orivschoolv but would not- be* .used again in the k current ^course." * This 
suggests that many courses, are structured so as >to exclude distributed 
t practice iri\the application of newly acquired skills, and could account for 
low retention' and: transfer of A earned ski TVs to follow-on courses and to the « 

; job.- , : , - _ : '-J- : :\ / \, . / -., y 
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: j •'• : ■ - : •>..,; •/■'v .' • 

An inspection of. materials revealed that self -paced materials appear to' 
- be better-written than materials for conventional instruction. None of the 
conventional materials passed a comprehensibility check- while 63 percent of 
. • :the self-paced materials' passed this, check. Long sentences and high reading 
grade levels, were the major types Of problems found. 

CUES. Thk.data |n table 9, suggest that many of the opportunities to present' 
cues' in instructional materials are not being used. This is true for 
* conventional and mixed as^tell as' self-paced instruction (appendix B). In 
' . the materials used, opportunities to provide various types of cues -were used 
only 38 percent. -of the time. The yse rate varied from 14 to 62 percent for 
. /individual courses. While the skille d instructor cnfjld hp rnmppn<;ating fn r 
. these deficiencies, the. instructional materials.' did not support these 
, functions. However, within self-paced instruction which is almost entirely 
dependent upon the instructional materials* to present new information, an 
. average use of 47 percent of the possible cue functions were used. For 
individual courses the range varied from 1 19 to 86 percent of these types of 
cue functions. ". : . . 

Observations; made while collecting, these data provided insight into the' 
way these cue functions are being carried out. .For instance, "courses vary 
> widely on how much printed information a student is given to provide ah 
overview of the course. While 43 percent of the conventional .instruction, 
"courses provided students with course outlines, schedules, student profiles, 
lists of draining objectives or other documents that summarize what will- be 
learned, in the course, only 10 percent of the 'self-paced courses ,: provided 
/ these types of materials tp students. ■ .k " 

: * v . '* '•> •• ''• ;. • » •'-'••■ ';'•'' ' ■ ■ •• ' ';•?'/•','••''• 

. The instructional materials - in these courses made wide use of both 

I , Words and illustrations; However, there were important instances where 
there is a mismatch between content and instructional material. The most 
' significant of these mismatches' is in the. teaching of procedures. Although 
performing procedures on equipment reauires locating instruments' " and 
>.'..> Tsontrols... and repositioning -of ' controls ^based on visual cues (all tasks. 

requiring' extensive processing of visual information), procedures were. 
- ; ■• generally taught by having students read written' steps. Consequently, 
materials to teach procedures for hands-on equipment operation and 
.maintenance were judged to be the least effective of the training materials 
reviewed. • * , ■ : > 

. :. y In general: • ' ' .,v. : '' 

'. • locations and identification cites were presented by words 

t •. _ " ■ ,' . "■ . " .• • ' * ': 

' • procedural steps were frequently unclear ,-. 
• • safety . practices were often not explicit ; > 4 
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. demonstrations of procedures .were, not , -included in self-paced' 

materials'* " . ■/ ..;'•' ' r,: •' 1 '/ 

m[. group-paced demonstrations were difficult to follow, v / V : 

.-.Other types of cues can be used to make it easier, for students tor learn 
and recal T. inforrnatidn^ Included^ are the use of " mnemonics (memory aids) , 
dividing information' into easily recallable chunks, the use of memorable 
graphics, and .Highlighting key . words to . emphasize . those words that when 
recalled wi ,11 aid students in remembering related^material . 'The Analysis of 
instruct ipnil materials in thi s study indicated these , techniques are 
inf requenVly-used— in-^ — ,. ; \ —. , '— — — 



PARTICIPATION. Instructional materials "were "a Iso, examined foir conformance' 
to eigh^;^ of *. good practice jftaterials. (se,e appendix 

].B) . J/frth^ Portly 38 perxetitt >bf/ jthe recommended 

. exercise techniques were used lit; -the' typical courses . . The mixed courses 
averaged using percent of the. types .of exercise' characteristics whi les the 
self rpaced courses used 63 percent. Individual courses' ranged, from 0 to 88 
percent for convent; ion a ; l riinstruction,' 13; to lOO-'percertt for. the t mi:xed, and .0 
to 100 percent for *^e;%elf ^paced^/ ' ' V ; 



V Observations pf good "practice and n/tes oil ■ hbw to improve .practice were*\ 
made during the interviews.; Perhaps ^he rifiost ' useful practice technique ^ 
observed was the simulated job shop\used at the conclusiQn , of several of the'" - 
courses. /.These shops provide opportunities for students to practice-, 
performing the job they will, be assigned on arriving at their new duty ' 
station. In th$ simulated job. shop students have; the time and' resources to ... 
perform representative t&sks, and to continue to»'dd thi until certif ied in ; 
this; performance. 'V- v ; . ; 

* . Distributed practice contributes to ' detention and the; ability to. employ 
the school -learned skills on th# joti. In some courses distributed practice 
'occurred Without being* planned or supported with special materials. \ Courses 
s amp 1 ed -rarely contained special materials for districted, practice. :, 

REINFORCEMENT. The instructional materials for conventional instruction- ■ 
us,e# 36 , percent of the types : of opportuni'Fies to employ reinforcing 
functions, While the materials for the mixed courses used 50 percent and the* 
self-paced used 45 percent of » these opportunities. Individual courses in 
all vthree types of' instruction' varied ffok .zero to ;1 00 ^percent employment of 
these functions., B The primary, observation concerning" reinforcement is that 
littje; attention., is given to its design' and schedul ing in . Nav£ training . 
materials;. > ,;" .' " • *, l 'v . - ' - / 

FEEDBACK. -Spared tests are ; t an. important form of feedback - to students on 
\their perfoy^wmce in a course.. The extent of 'feedback provided to students • 
in a course can &e v estimated by the types and "trie frequency -of tests in the 
epOrse. In, terms of "the present ..study,, a comprehensive testing program 
would score 100' percent on the:. xise of feedback opportunities.' - The actual 
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scores were = 42 percent vf or conventional, instruction, 58\ percent for mixed, 
and 60 percent for self -paced instruction. Individual; course scores varied 
over a broad range*: For Instance, individual self-paced courses scored as 
low as 33 .percent and as high as 92 percent, * . ' . " ' 

CORRECTIVES. .■; Printed* directions an what to' study to overcome deficiencies 
discovered.; through testing did. -not" vary greatly across the three types jDf 
training programs. Conventional t instruction used 52 percent of* the types of 
opportunities, while-mixed and self-paced , instruction used 70 and 63 percent 
respectively/ , '* : • * - "•" ' ' 1 

-EVALUATION t)F-a^FT WAVEDTRA 11 \ v ; / " V — — - 



v 



. Tn general, conformance, to current guidelines in ^draft- NAVEDTRA HOB 
would correct many of 5 the coujrVe^def icijenci.es identified in this study, 
' -'However, certain elements of quality .instruction call for guidance not dealt 
v'wItK adequately in this draft instruction. This 'part of the /report 
; documents an analysis" of ,thp draft NAVEDTRA HOB directive for thef purpose 
of identifying areas where additional guidance is needed. . .. > -> 

With^ the "help of CNET personnel who prepared the .draft HOB, the 
contents*/ of the instruction were^ compared with . the elements of quality '". 
instruction .on. the structured interview for evaluating. /instructional " 
materials. In this manner a ^series of deficiencies iV draft HOB were 1 
identified. " . v ■ - ( ■ • / ; . / 

PREREQUISITES. The -.instruct i on dbes. hot provide the ' necessary requirement _ . 
or /guidance for designers to properly' match materials "to " student prior ~ r: 
< learning. It does net . require that: r C -Vv. ; d \ 

\\ * . pretests be used to determine if prerequisites; ane known v 

• ■■ . / :f prerequii5ite^ .for a lesson" be'taught i n /'a » previous lesson , - 

- . ■. ■'■■■■■-"■:>: ■' v.' ■ - / ' " ;V - 'r--'"""^- :. -' 

a 'specified reading' grade level - cornprehensibi.lity level be 

;': maintained ■ . -".v ' '*'" • : ^ • 

^"v:^;* ./-v. s - : ' • . • .. 

• ''>■ advanced organizer^" be used to. relate/ Brevtous learning fee a rlew, 

".. ,/'""\ , learning task.; • ;, : . ~ ; . . : > * . \ 1 ; . 

CUES . ; - Wh i J e many of . th e CUE-oH ert ted^ reqai rement s are spelled out Mn the; 
. proposed : instruction, there are areas . th^t " need ^ to be expanded. The '■>' 
instruction does not 'require: • .* '" ' ? 

course overview* documentsbe given so that students 1 understand 'the • 
content arid flow of theitourse and .cajr : track:. progress through the./ " 
cpurse , . . . / . ; '•" . ' ,'" -y':- ' r ' V.r . : - f. v ■"■"*/ 

appropriate, communication channels be- 1 used; i.e., verbal, informal . " 
tiori with words antd ^isual -information with grip'hic^ . i 
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' • : demonstrations be used* where .needed and standardized through the 
' - use of .instructor guides. V,:., * ' t 

• ■ • ■ • ' ■ •» ■ * ■ *' ■ *\." ■ . 

PARTICIPATION. Practice of -newly formed skills is necessary for ^jeVr 
retention and usefulness on, JHe job. Whil|, the instruction calls rfTr 
practice, the requirement for pf*actic^v ; and its support needs to be improved. 
The^roposed instruction does, not require: , . 

• • . students to distribute practice over time with material^ designed 
•* \yp support distributed practice. " ' '• • • - 

REIKFORCEMENT. It is necessary tqf reinforce the acts of stTidyinlT^rid" 

acquiring skillafl performance,.. Although' reinforcement is essential to 

learnjng, - it is not addressed in. the. proposed i-nstruction. Specif ical ly+ 
the proposed instruction does hot: require: 

instructional materials to contain statements ; or events generally 
. known to be reinforcing at appropriate. points, in the instruction 



V 



V s instructors to learn ;skil]s>in developing: -reinforcement menus for 
. ■«,' individual students^ 'assigned to a ^ : learning center , with ol ear 
instructions oh how to shift reinforcers, withdraw : -;the us^ of 
\ 1 external .reinforcers^ avoid satiation .of effective 
■ neipforcers: ■ " ' S&V . . ■ ' »■ ■ 

JE^DBACK; In addition to using tests to . determine are 
present In students beginning an instructional module -o/- course, the >dpaft 
■HOB does not require: ; v;. . * '* <m 



curriculum .designers to consider alternative methods of ^rpviding 
• formative evaluation to learners, 'such as oral examination antT 
: ; discussion with prides or advanced students v ,■ ' ' 

* .. • . course "'managers to maintain rel iabi lity arid content, validity data 
on formative, and summative examinations. ; v :^ . 

CORRECTIVES. The 1. draft liOB do^ not describe, procedures'. to= be followed 
%hen a student- is unsuccessful in either fomatjye or ^uninative exam 

The implication is that .Such students go-back, and restudy the same, materials 
vusing the same methods, that*. lead" to -.fail ur^ tjiev first time. While/ the 

NarratiyeJ Summary, and; Programmed .Instruct ion' hihj v at alternative 'learning 

they do ri ot provide - for a* new, approach tq* teaching the; subject, .with new. 
'examples. : vThere ^ire. ;no provisions for*- alternative .ways* of attaining, the 

same objectives. *. >. . . \ • . " ... 
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SECTION IV 
DISCUSSION 



■■"'■"■* This section discusses the findings of thS study, First^n apparent. 
contradiction in, the. data regarding the r^lat^ve effectiveness of 
conventional versus self-paced instruct.iqn is analyzed, ; Next,! a summary, of 
findings-* related to elements of .quality' instruction and variation ; of 
Individiralized .instruction is presented. .Cost data ar^.thert discussed in ; 
terms *of' their usefulness in assessing the efficiency of self-paced 
-in st r u ction. '- — - — ■— — — — -■ — .. . ■ 



The intent of s study was to -examine the variations in instructional 
practices in the courses classified in NITRAS as individualized/ instruction. 
The data in section IIHshow that there are indeed differences in courses 
but. there are few .distinct patterns attributable to. ..instructional strategy; 
i.e.,- conventional versus self-paced.. \The differences in these courses are 
primarily accounted for by instructional practices which can be for the most 
part employed irrespective of strategy; that 'is,* the' degree" to which they: 

- use* good -learning principles. As defined in this section, II has three / 
^essential ingredients; (1): releasing of time -constraints, (2) choosing / 
^instructional - media, and (3). adjustment to skill levels and learner 

characteristics. The NITRAS categories of . self-paced instructieh " '(P)., ^ 
computer 7 managed -instruction (C)/and,a combination of -the two '(B)', depend on ■ ..: 
.self-pacing' to determine if a course As individualized; This cari" -lead/ to- 
'errors .in assessing the effectiveness of truly individualized instruction. * s " 

., , - ; The d'ata ..in section III also show an apparent discrepancy' ^n^findings - - 
resulting -from the use of the quality of instruction questionnaire arrttftne 
''Structured interview. The results from -QI general ly . show th at convent i on al > > • 
iflipuction is superior to SP in the provision of: 'cues ( ^pa}^l6^iElM^^^>^>^ 
-/rlifel^rcemdnt. -Conversely, the. structured interview 4ata* show that SP " is 
"superior to CI in' all Part of tnte: discrepancy' may be 

■due?|tp .the. fact that' tKese time measures^sk' ^^ f or different information. The : 

- QI. asks f 6r. ; . perceptions: and judgments about instructional < practices" while . 

' the. structur'ea interview is designed to -assess ^instructf6n^l : niat'en*als/^^? 
-Examination of table .9 indicates that on a relative basis SP is .superior : tp ; • 
:CI. On an absolute basis, however*- average presence .of QI elements reported 

y for material is f arjfronr what could be "considered . acceptable, in any of the' v 
VARII. CI measures^ranged- from 20. to 52 percent, the MIX range' wa's 30 to 71 " 

. percent, and. the' SP range was 45 to 63 percent. One -interpretation of the 

. superiority of CI over SP in the QI questionnaire's that in a tonventional; '•••*>** 
environment, instructors, have more opportunity and. appear to condensate . for 
.deficiencies -in instructional materials. . This; interpretation is .< 
strengthened by the results xpf. the Johnsori and • Gran am (1982) -study "Which > 
showed y earn i rig center- i ns triictor s . sperfti most . of ... their, time i h short -and • Z 
routine transactions ' witn ' students thus precluding their Opportunity to '■" 
provide other; cues, participation and reinforcement, f Tfte , explanation '." that V 

■{ instructors wil 1 compensate .^fpf '^urr i cula* defi ci enc i es i s further supported •, 
by > the data in' - table , 5. These .data .'show that CI courses'- in •" il 
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the- present study had, lower S/I -ratios, /therefore, providing the chance to • 
compensate for deficiencies in jrnaterial.s. ■ : . 

Additional /indings"" of " interest are . discussed^, in the following' 
paragraphs. They are organised* by the. six elements, of quality instruction; 
so that they conform to the approach and analyses of thi s study. 



PREREQUISITES 

Cv' 1 Qufesf i onnaire .'respondents in " aj 1 ^variations (*f; II perceived 'their 
cotrses as doings ari- adequate job adapting' to individ ual ^diffe ren ces ■ inyp rior 



learning, intel Tigence,- and attitudes about, school . Studgftts in CI classes 
Had the best attitudes about school le^rip^g at the start of their courses 
and are ^ peffceived as performing at ^ a higher level than their abilities would* 
alTow. Students in the-rnixed and self-paced classes felt their study habits 
were most , improved. Although students, instructors, and managers : vg^e-;th^ij: :; 
courses relatively high grades ^ there were still instructional^ 
problems., the single lar,ges*t cause for poor performance, in all 
instructional settingsv is tfie. mismatch between the skills, knowledge, 
aptitudes, and attitudes a student brings to the task and what the designer **' 
b ; f v thV-t^ brings., \~ 

f ' 7 ' - r ' \ 




. Questi onn^iVe . respondents f el t'th'eit mat en aTs were 'well presented '.wi th : 
Clear objecHves and maps \of wb$t the students are* to do. Instructors are 
perceived "as expert, with those in CI courses lecturing "more,, better holding 
the students 1 attention, and spending more time helping individual .students 
^h^ in,-; other -courses.: - Analysis of the structured interviews revealed 
"instructnonaT materials need more ^clear cTff^tions, variety, in presentation, 
high-quSlity graphics, and, guidelines to Instructors on their proper, use. ^ 

t^nciPATio^ ; / . .. ', 

^ . Questionnaire;- respondents in aTT levels of 1 1 : fee] that students spend 
rpucti of their \day in constructive^ activity. Those' in conventional classes 
; ™st^ of that which they are -to learn, however, and 

are fnbs%: 1 i kejy to ptSt i rt ? ^im€ '-beyond the scheduled workdays -Those in CI. 

lesson materials and instructors are more likely to 
hold student i nterbst , Iceep them, from daydreaming , manifest ;apprppri ate 
teaching methods^ and elicit more practice than do thpse ; in^^ 
/The tjme between presentation and^practice was seen is shortest in the SP 
^ The instructional materials are weak in distributed practice and 

i n : pro vi di n g student? apportion ity to practice newly-acquired job i lls in a 
simulated 7 : or model . work environment. ' - - : a '-\ ^ 



REINFORCEMENT 



J. 



Questionnaire respondents' in di courses are seen as jtiore enthusiastic, 
receiving more recognition, for their efforts, and more likely to s work with 
and for" students than d6' those irr SlP courses. 'Students'-in CI courses are 
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most lively ...to. see the importance of their learning. for.,.a r mi>Jitary; career, 
and .are' most ljkely > to recommend their courses to '/ other .students.- 
Instructional materials' were^fpund \p* have Tittle built-in reinforcement. 
This could be -due to . the absence of the topic from NAVEDTRA 110A and ■ 
previous guides. .. . * "'■ t * ■ 

•FEEDBACK , > ^ , . "'■U / " v /f '" # V'^ -v.\ 

• Questionnaire, respondents feel that there i 5 more : testing .occurring in ■ .[ 
the SP courses than CI' and MIX, .but -.that 'there., .is jftpre< instructor \ 

..Ihstructfbnal materials; were,: Adequate . in providing - feedback; however, 
reliability and validity *of 'the ^feedback mechanism Was not available for 

vstudy; '-r -~ ' ya ,. > • • 

CORRECTIVES -■ ;J V*. ; .V ,\v "' 

. Questionnaire respondents feeT that Sp and MIX. students^are^ore^ 1 ikely 
to take corrective activity within .'the classroom or learning ceriter and less 
likely toK be set. back" 1 6 a later: class than those" .in' CI. .The availability 
of restudy materials following failure- of an examination "is, perceived as * 
being high in all Teve.ls of ,11, but highest in the mixed II courses. >;'•'•'• 
Instructional, inaterf.ajs generally build corrective procedures into classroom 
and learning center -practice,' but •• data from . the .interviews; revealed 1 ittle •' 
use of peer tutoring or instructor, aides in ttiis ph as e of Tnstr Jet i on tf&J?. * 

COST OF INSTRUCTION '* \»'<\ .-V , '."V ' 

. - The, study results indicate, that the direct costs of ' self-paced . courses » . c 
'are approximately / ;one-thir.d . . to . ..one-half • the : costs*' 'of • conventional 
-instruction:.' depending 'on the ..manner in which costs are calculated^ This, . . i 
however, does not include, curriculum development costs. An earlier report* 
found the cost of graduates higher in SP than in CI (Dickinson and Swope, 
1981^?u :|f both, sets.; of, data are to, be ..believed,* they -carry y different. '-..• 
implications for*. CNET policy. If average cost per graduate': is indeedvHi^fer 
for SP then this should certainly "affect decision's, about undertaking the 
conversion of existing CI or new instruction to self-pacing. If, however, •« 
the direct cost of delivery is lower for courses already individualized then ' 
decisions can be made relative to. the continuance^' siich/ courses. In-iSb'th 5 ;.^ 
these, instances equal effectiveness under CI or SP • is assumed. 
Unfortunately, these and similar analyses' were- peripheral "issues to, the main ' 
objectives of the studies in- which they were embedded. If these res ii It's can" ' :i 
be validated through^ more detailed analysis - . of • -^exi sti n'g; cost data, they 
would provide the -basis for policy' regarding the use of instructional' 
strategies. 
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••/'.-..' • * : SECTION. V i .V v- 

^ • : V '/'■.'/ ;- : CONCLUSIONS AND RECOMMENDATIONS 

. ConclMlbn^ instructional practices, in .self- . 

ijace^L computer-managed' mixed and convent defined in NITRAS 

are , % provided . here-, tqgether with recommendations, for improving the 
maWagpent , , and v conduct, 'of those courses and for .modifying NAVEDTRA 110 ^ !> 
(sei^s.) and ^re-Tat ed instructions. , / ^ v ? , : f 

CONCLUSIONS- ; \ ' ' : — f^TT^ ; , ' /-V:,/ ^ ^ — \ V - 

■ : /l. IndividuaTized v instruct ion, .defined: as including (1) release of 
time constraints, (2) - choice of instr^ct:i6hal me'dia, and (3) adjustment to: 
skill levels* of "the vi earners., rarely occurs 'in the NAVEDTRACOM. Most' 1 
/courses categorized as "B," n P," br/'JC." jn' NITRAS contain some II, primarily > ; 
rSlea^e of ./time constraints (self-piacing) : . .Other aspe'cts of II are present^ 
in « these courses - in,; varying, But" in r Insufficient ^ 

categorize the courses as "individualized instruction." '// "' V ^ : /V ' 

•s^->" ;2;,/ Jn /.the 'courses • examined, ^jual^ty^ of' instruction elements: ^cuesv< * 
participation,', reinforcement )• were perceivfed- l>y questionnaire respondents feo . 

/♦occur with moi% frequency, i n d th;an /in MIX; or SP courses .^This may /be t due^ ■ 
to the lower ^ student/instructor ratios.- if); " CI • > courses;: all qwi^ r greater ' 
opportunii^ " "fipf /'instructors to '.c6iii|Jens.ate. fbr^ty^ 

no fjqm\Jjf i nstrijctoort Vexaml ne r d -ticweveir > ' Were any of r the elemients present 
f jtn thev degree rtecess'ary to qualify as II.- Although the " attitudes ,an,d > 
' perceptions ■ ?of £iudeht$v/-4 were positive toward. 

a t lT aspects of instruction, regardless of m&hod, examination of, materials 
and vfsitsf to 1 dlassropms. and learning- centers failed < to validate this 
optimistic outlook;" \ , ! v M ■ ."/ ' 

/ / , 3^V ; Both . student>^tid * nonstyident direct costs tor the SP ^courses in ' " 
. tHi.s ; stuHy w6re 3W-58^er^t of thbse required for the CF courses: y t 

:i ? Seif^paded 3 . instruction ^ is ' heavily dependent on, written v - 

instructional materials. Consequently, the proper design and/ use of tl\ese/ ' " : 
materials is even more crucial than for CI courses. Self-paced mater^i al s 

examined in this study were super tar to. those in CI and ,MIX- courses; / 

-however, on an absol ute / scale of ad equity^ instrnlcti on al mater i al s wef^e 

found to be def i clent . ;ln j al^ ^tfYrefe'/typek .. of courses. Primary areas of \ % / : 
deficiency in $P materials include orientation to the coyrse, teaching' qf/ 

^profeedures^ r use of memory-aiding techniques, inadequate d^poft unities for r . 

^distributed practice, .:and^ limited use . of performance "t est i ng . / / «■ 

~5. * Despite the ^uttjtual ly exclusive 1 abel s >appl (ed to the . courses in 
this study ("P, Hhd "B 1 !) most' ^ 
;^t^ategi.es to 
rrtbst: "cbiirses. 
wit+vih^ 

tasks and . that , a sii 
tot all . tasks, within a gi 




liriijTe ihstructional, strategy will, probably, riot be sui^afy/1 e: -f. 
in a .given: course. ;; , ; : /- l ; ■ ■ -'^ , . : //•/;;■.: "• -s * 
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" -6v ; Instructional strategy (II vs (y). fs -not ^tbe^determiriing factor in r .-*«? 
overall course effectiveness, of current tfa# courses'.'-' 'Rather; ■effectiveness^* .-" 
• i-s, ^netdiated by.. the extent to which good instructional: practices -'are used 
within the courses, and ultimately determined v >y, the degree .to which proper' 
instructional objectives have -been defined and met.- " Good 'instructional - '-• 
' -practices. pan., be employed within various instructional strategies. Since it -■ 

l'S -possible - for prcperly . exefcuted^ II and *<&■ to be ' equally' effective, a " 
i. choi ce of v strategy should rest -primarily on '-'the relative cost efficiencies ';• 



-of the two approaches. ' - -, ; - . V. -->,_>-. ,.' 

RECOMMENDATIONS "' Z V.^ '- v l V 



'T '-'A V Revise NA^TliA; : iip> ts«etlJBsS' : . :ahd^ra^t^ : , d i f-ecf "i vtfeV \to -include the 1 •» i 

• following guidance: v ' ' v' vv' ■• •v?''*'' . ' : : - - T'-C - ' /*.'.• ,■ • '*',!',''.■ 

:v < : :--*'lr ; v''a»: /Use 4 ; pretests to; aetefmine' ify a', student, -meets- ''-the-r entering . V 
<.,r.equi£eme'nt.s and "to diagnose; specif ic ; 'deficieYicies..- / V - ^-'- <■'...--:-'.. - . " 

\ > . ' • ' - i ..^-Vv-'i •■-■^'V ' • - 

; • ." . •< f r )" -Expand paragraph "2 .2; 4yl| Pretests'; to- i ncliide the use ; : • - ' 

rffiF -pretests.' to ^determi ne> that - students have required .entry' behavior jv'4 . e'. « - 
?.^.^^.^^-t^ag': speed and accuracy, before allowing them ; >to '"enter the /•"•'"• 
-main body oT-the courses i • j r . ,*'•••" v. ■ • 

v. vv' 1 . .: '' '.:« ^ - .' - •; ' .-• * • • 

^> - V ,b.> yUse 'appropriateTy-designed, training - materials to. remediate "' . 
. those students* who do not meet <cr iter iqri en Pretested ehtry : sk'ills. ,. 

■ (1) Expand the Student Remediation Guides for u|e by Learning .V-V- 
Center ..Instructors, , add i ps, corrective procedures for 'l^^diatinq requiredi 
entry skills. . .. " .-.-y ••* V: 1 * r.'%V. 



; These mdduTes 



^^fe'^e-ate -Hehliry- - sK..t1 -Is •'cprpect We ."ino<ru|«is<for- student* use. ^ 
es- shou ld teach the> math ski TTs ' ,andl ot'her ? spedi f ic entry level ' - 
skills' required but?.. pot taught :in the- main Course. Use variations on the ' v ' 
corrective math module of the BE&E ."school as a model. ^ > • ^' '•" -.V'-^'J.v'-v.j":"'-. >■ 

• - • .v* : • 

/ : *'V;.. C J:'^ W^-* '^uf6'op§ --'-"j:^.,,-; write module, booklets according to VV'V.'. 
guidelines' ort : readability and c'omprehensibility, with-.,the v goal of ; mak^fig' it ^ 
easier for targeted students to understand the, text that they read. . ^-> ; -..\ ( •; -v'*; 1, 

^ r ' (1) 'Expand ,-paragraph '3.;$.2.1. .t.g incllifde g.ii fdeW pes' r for 

writers to: <{a), 'Use, controlled vocabulary, (both* common'; and .•technicaV 
/specialized words) which can be based on carefully developed- lists (see " 

• Kincaid',. et al^, -1980), (b) avoid ^Words and pbriases designated as awkward, '• 
,{m ayoid;. -awkward , 6r* difficult sentences, and ( d ) wr i te at ' a^ -readab i =1 i ty 
grade level iijatched to students' reading ability. , . J . " - 

■L-. + 'rl^^^W? f-^Kl ,: >t»rat--i:l^'checks' "Wk .ai^atlcalily made by.;'' Using v 
,t.he;. :^ytef^ffe«4aii : 1;Hftr;-.f<Mtiftg*:-:-Systeiii (Kincaid; Cet Va-l..^ . 198Q) ;;" *'. The 
development --and; -use of. .readability 'and ■ compr ehe'ns ibi 'ii t-y guide J i nes ts; : • ' ;>: 

• required by .OPNAVINST 1510.11',: ;€nlisted : Fundamental Skills -Training - of 19 ::i ■"' , 
August 1982! ■ ... ' '■ PC ■ " VJ - '"' '"' " ' .' - y... 



■'•«!■ 
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• Y d^T^qvi^ ' -i5 r tudenjkli-- ' : wtt^i „ ii;.* .c'i'eai i : , l 1,I desXjf;''1 p t fort rt .flf jwh'at they will 

. '.'-be able . tS -qo' at the ^ompletigin,.; r of the course and- hWihe ^course is 
. . J- r; organized to help.:- Him ^qHlev 3 e'.;thii goal.Y ..(This? lis . bejng accotfbl isW within 
' ;.. ; conventional group-paced courses!, but ,n.ot. in ihe self-paced courses i ) " 1 ^ "• 



;vy sequence of ^e;ssorr>M<ftL ;tests>fa^ ;ah estimated 

eomp jetton time,, (n-angej for :ed£h g ;fniry*J'; L :.f ; ; v 'X - ;y ?:-V-;"" >- 



' : .< •♦>. * v (2); provide - students ..with- a copy of "the '^luderit' Profile, 

course. • ' •• v . ■ .... •. , v "-> ' ■ '" • .'• ' , '•. 



e. Provide directions within NAVEDTRA /ilO (series) r instructions 
for.: •■ •'• V . * ; .• r _ ;. ;., i ■ '. sis ■ 



/ . - r ' . : . ; - ^ ::".fih ^uidio^^traihing. -...^ystems designers ' in " identifying. 
. .Jeav^ing, tasks* •t>'esv^bcbmp1ii^ed;^.^r l 6i!ip instruction 

' .• : ;; Y.'(2) 'incorporating Outline' of Jn^tructfon/lnstructor Activity* 
pages In th'e^Learn.irig Qenter Instructor Gufde" to , support instructor led'" 
^ lectures,, discussions^ , Snd*;d^9ifs^rations''-fo'r, selected topics. ; _ . r , *'. , . .- 

*!*■ ' ' .. Y * . '(3| including information in the Learning Center Instructors. 
■ Guide on -how to schedule- students who" advance- to" appropriate, «z6ries in the' 
.curriculum -to take part, in group-paced "'•activities ' <Y .: ; 

.Y . '"<•;• (4) ^guiding ; ^traimng" systems. 4 ^eVi-qner^'Vi tf- '-jtKe layout 'of 
classrooms suppo»*t^h^ bot"h^ : '' self-pac and 
incorporating these, layouts in thp J[rfstructional Management Plan., ^ , y , ; 

; _ ; , f. Use text-graphics .pages," to ' ebrivey .AifsuaT information ' that 
cannot be efficiently presented; >ith; ^wor<is.. ^h^se v ;bigh:iy: illustrated pages 
, ... are, especially useful . in>:aidihg,> thfe, ; student- M" n ' : loca.t.i ng -.componVnts. on -a 
Piece ' pf equipment, ^p lacing , switches- and levers Sn, : i -'prescribed vprbsition, > 
and recognizing signa-ls'or- system responses. " - , \ ' 

' • .' :• . (1). : Make- use of. text-graphics.. pag'es in teaching procedures , 

or in presenting procedures'' to be, followed ^ in laboratory exercises involving 
* the operation or maintenance of equipment. lUse the TAEG format model. for 
.Procedure learning (Braby,, Ramel, and'Smode, 1982). ' i 

.. {'if' g. . Provide guidyiines^o ."instructors On how and =when to. conduct* 
: .demonstrations^^;. I^ 

■h.,:- confusion in £ty.dejits' • m.1iids:*-'.>:^einQni54^^fd < is' "a^e' an- importi^t jriSde of 

" , training in group-pacedi ' a,nd .*B^xed^ 4|}struc[ilW -;and should 'be hand-led •. 
. .•/.skillfully. . . • .,' ";/■■•■> /I,' ■ ."••M-^C'^V; :•••/.'-.*• 

'y'iiC' ;. " ^ r -;f- -\ -O-V ' .'. ....... ; .>, ; - : .'„;, 

¥;> : X;-v* ^.V (I).; Give direct ions.. on how to conduct a demons'tratton within I 
sec^i*n , providing . guidance- to instructional •systems designers on 
. creating*. Instructor GU.ides.. : ■ 
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. ""V . ;\ .■ r ,- : - »•'-;■' .-...I. v •■ '^y : ' .. ■ - '^^'tX;v^ 

/ - * (2) • Require that an Outline • of Instructi oh/Instructor * 

Activity Page be created ; for each major demonstration; .. lil the ; Instructor ? - 
Activities column of "this page •■spell out exacts jwh at '• -tjfe-ftlWtruttor lis ^bV- 
do in this specific demonstration. *• V'-V;'.' 

'■■ ' \ h..' Prpvide Retail ed , "• guidel i ries ' to Jins'tructional systems,. • 
V des igjiers 'oh how "to identify '. 'when- • tfistri buted>;pr^ct.i.ce js needed , . and hbw v tq 
. . create material s to support distributed <: ' :'>''-•';,,'*; '■ '■■ C 

: .'-^ ;> ; V :i 1 ) . ; Pi r"ect ; ' i nstruct i onal ' ' sys.tenfe : ' ; designers- 'to'- '- build ■* , 
TTd-lytr^buted p r ac t ice- exercises for th at. sub j ec t m att er not no rma 1 i>; us ed 1 n 
v, les s/6n's. .^ubA^^|}'t:/;^f.1tS>'. , f-n1i1'&1 presentation and " add this guidance* to r 
; ? NAV|DTRA 110. (seriesT): under the ; ,headinrf "reduce forgetting - by . providing * 
..•periodic oppqrtiinity i to.'recal 1 and practice i nf r eq'u e n t ly v us4id 'm at e r i a 1. " >• . ..*' 

: v' • i. . DTr'ect; t instructi onall syslems - designers Yto p'rbvide each- 
student .with the. ^opportunity to practice the newly^cquired job skjlfs.Jin a ' 
; •■simulated or model work environment. Jhi'sl should .be the fi rial phase xjf job 
; training^, and students , should not. be allowed -to. leave 'until .certified"* that 
v they; can perform the. work described kj^tbje^itudent profile. • • ..'.*/' 

(1) Modify section 3.6.2, Instructional" Materials for ^etf^ v ^ 
paced Courses, and .section 3.6.3, Learning Center /Instructor 1 (LCI) .Guide", of ' " 
^AV.EDTRA, 110 (series)" to include" directions for^. designing exercises' in "• 
-simulated work - environments, and guideline's to :1 nstrijfd^Vsl : 4hdV^udWts for .**•'•••" 
carrying out these exercises. Sample material s^should be\ iritTtided. "' / '•• 

!?' " ". •'• ' ', ■'. u. **'-.■', ■ • '/*"•*'•'. " 

(2) Modify paragraph 3,&.4'.5f -final Comprehensive test, of > •• \ 
WEDTRA 110 (series.^ to, specifically state; which -part of. the comprehensive 

■ teSt' w-fy, when^ possible, i4 be a jobrl ike, performance test in "a sWlated or , 
model, work^environmeht, and that students not Be certified" yntil- they ..pass * 
this performance test. •"■ f}'" '• • "" > 4 < " > t 

.... . j- More alternative methods and materials need to be developed 
for. iL'CIs to prescribe for students failing to s,how mastery under the primary V , 
method of instruction in course. ' SMCh^. r cbrrectivei 1< ' -should' '''include • 
sound/slide packages, books, and articles keyed ,to particular areas Qf v . . 
\Studeht difficulty in the course, microcomputer-based CAI diskettes,'"' 
counseling -by subject-matter experts, assignment of advanced" students as ; 
tutors, and a longer school "day. v V % • 

". : W k.. Provide instructors and supervisors in prerequisite courses: 
with .i\nformation concerning former -students' performance in upper-level* 
course's. '1; . ' . .fc'-'-rf 

: 1 v>%aR instructor accountability^ tor students' •« 

academic: succesls into, the training system.. Make instructors' names a' <; 
permanent part .of every student' s record. ,• >. ■ . - ', 

, 2... , "Compare the effectiveness/efficiency of the 35 Navy "P,"~ ,5 C," and 
"B." courses; ekamined- in thi.s study using, course specif ic data-from the vTAEG ' r - 
incr.emental costing -;mo.del and the CNET training appraisal s ; ys tern.' Identify 
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V arid ,i correct ; course defi c i en'ci es ; in Vccfordance with .NAVEDTRA 110 ( s er ie>) and u 
<7 ..reyisi^ v ;^''''"'.; ' ' . ■'),-,' /'f ^ 

: >: '"V'-3* 'Judmduct a: controlled '.study pf ^the^ relative;: effectiveness and 
M efficiency -'of. the,, conventional.- and -individual fze'ck iVnstructipn -formalts. v 
• Select a 'moderate thr course* with suitable curriculum .for the - 
>" r . % , development \of ah idealized . -program ' of instruction/ under either -of .'the »■ 
formats. ^Ensure ^hatpth^ management are s6i table -for 
the ' assessment of efficiencies. vUse study ^OMtcorftes vtoi- assist with - the ?: 
_develQpmeat^ofLpoJ^ 



iv" . 4. / Use the projected : T as; a* majors factor in choosing 

?■ the main jhstructiorval strategy (CI Vs II) : fpf^a ..new course . When "chops in g : 

w whether- tp. change the ,4ns tnucti opal strategy of aft -ongoing cqurse;. use 

\ incremental costing techniques/^; : \ ■■• : , ' fc .' .\. . 1 : -.■ 



f 
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""' • .'. ; course description form with summary statistics 



: . ' • • & . ■ * „ 

This. is the questionnaire sent to course manager^ 
■ ■■■ in the ilrritiaT part of the study.; The response categories 
^ are filled in with summary >tatisfics f rom the 62 cours^-:^ 

;> shown in table 2. AIT. numbers -are fr6qi/encies, except 
. where TabeTTed as mean (M^ot*. standard deviations (SD). 




■;:'w- 
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'-• '. .. : , COURSE DESCRIPTION FORM .. . // > V 

The Training Analysis; and Evaluation- Group ,v(TAEG) -^ls^continuing i\ts 
study of variations in individual i.zed- instruction' for. the Chief of Naval 
. Education and TraiVi.ng (CNET).. Iri 'the present phase 'of the- project, .it is 
",. necessary to obtain^ a description of current practices-, . -Accurate responses' 
tiD-the-quest™^ 

the 1 earn i ng eriyi ropment of future; Navy men and women/.' : , "<■ ' '■- 

v- Please, have the' persons most familiar with your course fill, out the" . 
,i -questionnaire. - 'Describe the v course- as it; presently is bei ng taught; 
■ regardless of your plans to revise or change it..". 'Jitf to .be as objective and 
factual as possibj^ in your, , answers, ..even when thet question seems to require 
a subjective judgment.; "shouldn't take much more than a' half-hour.. 
Completed forms should be returned to: ' 

• + .1 „ ■ 'Director-.' ' ' ••• . - ' ' >• 

., Training Analysis^imd Evaluation Group ' 
/ Department of the Navy * ; ' •. * ' : 

• Orlando',, FL 32813 > ■ . ■ : " . ' 

If there are any questions concerning the project, please contact; 

- .;• .Richard fl: Evans '■ ; V- V- . . 
■ Training Analysis and Evaluation Group . . 

. Autovon 791-5673 . ■ ' ' . - 





(Titl.e of course .,) 


.v. " p ■* .. . ; 


(Catalog number) 




(Location) 


. (CDP number) 


(cjn) • , > 


-? ' (Type) ,, 


u*, . (Name and rank of person responding) \ 

" . ••' i ' . ' V ■ ■ 

(Job title of oerson resDOi 


(Autoybm number) " 

ndina) 
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1; i Do you recoVd ^contact hours, as .we 11 as calendar days to ; .• • > , • 

H* ; coiplet'iofi-.for'indi-vldua-l. jstudltattt* (CfrcTfe). .j.*;. v.- .>, . ;yes no- 

' ^ - ••->.•:•;:. ' . ;/.(34 28) 

; v ilf- Yes, where? are, these," data maintained? \- '»'. , 

X- f 6 1 Students' self-records. .. . : V ". % 

I li'-V Master 'record .'files r i n. the training office. ;-..;> n :' .VvV- 
( ~6 n ) Hard, cards • in each classroom;,; . ' • 
' ^ P j-5 ) v In a student academic record book.- > 
.. ( 10 ) v 'lh the computer. memory. " , . -"'.v - / 

iZE) -Other- (please *i*ist):^ ;. 



• (44-iy *. 

-2. "•- Is 'this- course in shifts?' If- yes, how many, per day ?. . . (,B#3) ;v> ; 

* ! ; v . * '*•>• ••' '-' ; -' " ""'"'^ - ' : • 

3, - How many hours in a typical class day "in; this course?.. -. '.■ (Mf=7.38) ,; 

•/' ;\;--r.,:-v'V- :v .. ■ (5d=.82); 

4. * 4 How many learning minutes are, there in a typical ^cl ass - ,. 
• r hour,?. •'. ' . . V. ...... . '.. . . ' : : ( M=51 . 31) 

5: Is break time included in "'your recorded class hours'? T x . .. yes no \ < 

■M v^W'. . : . : : - " (39 .21) * 

6. ' C^jrt individual students determine when .breads Wil l/pccurJ^ yes no , 

" : • -V''- - ? ;.^'V. : ... . ' "V v ]'•<■] (49 ..1.3) 

7. V Does remediation time -add to ^the recorded cl ass 1 ' . ,« ; I 
\\_hoursHif it occurs over and abovfe normal class time? . . yes^no- 

■ ■ ' ' ■ * ' 023 . 38) , 



8. What is the. average number: of working days .to V j : 
^ completion iof this' course? . . : . . X '/ . . >' ... .( M=34.89 ) 

- l . , : ' 1- r (SD=29.90) 

•* '.' ' ' .' ' \ , ' . ' ' ■ ' ' ' ' ' . v ' ' ■ ' ' : * 

9. Jfow many average working /days- do your faster, students : 

take ..to complete?. . . . . . •. ... . :" . . . .... : . . ( M=26,56) 

■ : i ' .. .' - f V .!■'•' - '1;.,...,. ■" ■ (SD=24.43): ', 

10l ' How many average working days do your slower students 
y take to ffnish?. . ... . . . '» . » ... ..... .( M=36.49 ) 

> i , ' SD=30.07 

11. How many modules (or lessons) are there in thi-s course?. .( M=19.00 ) 

.-. ^ (SD=19'.20) 

12. How' many days do students general- ly spend until attain- 

inq mastery on each module?. . . . . '. . .( M=2.31 ) 

• " , ; '•, ~v (§D=2.l4) 

13. \After how many failed examinations Would a student in this 

Course be (list the number of .failures next to any actinn): 

- (M=1.00) Given within-class remediation or corrective activity. 

( M=1.00) Given outside-class remediation or corrective activity. 

( M=2.13) Sent to an academic review board. 

: ' (M=OD Other (please ljst): " * 



'U^ '' • 4 2 
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14 What js the criterion for rnastery in each lesspn. or module?; ^ */ , 

• ■'. if • . 4. • .v . . . . • • » ( M==80* 66 ): percent 

' ;->-v; L,; < ; ;v ', . (5D=^.48); • ■/*; 

^151; About how. many students achieve criterlbn/pn"^^.-'.^ "C V/;" 

the first attempt in most modules or lessons?, . ' ;( M-72. 65 ) percent 

; — : ; V . , - ^ . v — (SD ? 32v6r) : 

; -. ,' * . ' ' ■' ■ v . ; .:;,V ■ >" ; ' : V-'?. " :: *; ' ' * .. 

16. - In^generalv ck); you ¥eet thfct'the recommendations ,■ ,\ ;■■ i v.j\ ..- . . :\ 

made by ^ ijistructors tp academic review boards, " With . : ;, . 

regard >tc£ dropping jor setting.; bagk students/ are . r , 
•■ ! followed? . . . ... -V »• v. . ; Agree Disagree 

v- v"' : - . " £ b '/'•-■..>' •■■ -s (45-. 4j : ,;, 

17. What is v your" estimate of the percent of students sent to';' \ 
;acadernic review boards wH6 are/set back?. . . . . . .( M=26.82 ) percent 

,, ; ( y • \ . ; . " •^(SD=37 ; .35) v -V 

18. . Estimate of the percent , of students sent to .academic ;/ * >: 

review .boards Who are dropped from school . . . . .(M-24,58) percSht 

* , ." : ->:■■ ■■' ^ ■ < %. N (^l49) ; . 

.. ;V _ ' 8 ■ . ; ^Vv^^i ■■>,/ ^ ■ 

19. 'Does this- course riequ,lre a comprehensive ■ >' ' • : 

end-of-coursi^examiriation?. i . .. . . . . . . .^.^. :^vV' : \ ^ ^ 



20. Can studentsv-fail ^tjieir* course because of a low compre^i; Vi : V'yV: 
.... hensiye exam score? . ^.yV; ,? .i^ s '\^; 



21. For the following, instructional management functions in 
the classroom or: learning center % indicate (by a check) 
how it is performed: ■ y 



( : 32 23) 



• .:;.vf^;<f : vf Instructor Computer Both; 
^A^signmeh^ (46 4 > ; ' 8) 



Compos trig exams 4 - 


(48 " 


■ * 

, 5 


2) 


Admin i s t en* ng exams 


f 48" 


3 




Scoring exams 


— (42 ' 


- "10 . 


• 4) 



Prescribing remediation or J ■': "p '■■ : % .".>•?.'•:/.>•;•. sV 
correctives - .; f46 :/ ^2 : , 10V 
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:',.^'7 ; v /22?„ management functions in the >• 

* - iaboratorv ;' indicate (fc^y ; a check)" how;: it is perforthed: •> . . <t 

^ . v, ,r-- . V^-'V/A : ■ Instructor. -Computer Both 



-Assignment of learhing material :;' 



Composing exams \? 


- - (43 ^ 




• : 4V 


Admirpitjer'i^ 


* < (44 ^ * 






Scoring exams ,.' 


(44 • 


• • 2 


* 4) ' : '• 


Prescribing remediation or 
.correctives \ ; \ .«» 


.(45 


1 


: 4) V 


Record^ keepi ng y ' . 


" 140' 


2 


9> : 



23. Estimate, the patent, time that each of the below is priisenjiA tfjis 
. • ■ course (must add up. to .100): . v ...Ci. ; \'-\' 



(M=16.03;$D=21.84) -Lecture. ■ !: - 
! 6.00: . 11.18) Discussion.- r , ' ,. , 



( 5.82: : 10.14) Classroom demonstration. • -v:"^;*^. 

;,i, iv? 'f 35.76: 34.96) Self-study of reading materials in "the classroom.' . 
.;: V; ': : , 2.44: - 5.ffl Tutoring. ^ r ' : 

(";--;■ 2;21;^ 4% 21) Films or. television;. tape's.. 
r ^ v, f' 29.90; 24.82) Laboratory exercises 'or demonstrations. „ 
■ .(^ 1.31; -l^fr) -bthefe(p1ease list^;' ^ ( .^.r 

."" 24. List the different. ways this course has to teach the same 
: „. • objective^ (such as the, summary, programmed instruction, and 

narrative mentioned in NAVEDTRA 110)? . . . . . . . 



(M*2.08: SD=1.41V 



25. Estimate the percent time spent .classroom _selffstudy that is 
' A devoted, to reading the -fol lowing ..types?. of- mater fais (must add 
up to 100): ; " ■ '••V.\.- v}''' ■> . 

(M=14.35: SD=25.27) Student guides. ^ • , S ;"V"' •. ,;. v .'\ 

( 36.61: 31.88) Modules with- summary, narrative, and programmed iristructibn 

, (such as per NAVEDTRA 110). , ' 

{ 2.60: 12.92) Other programmed instruction. -/. y 

( 15: 48; 24.411 * Handouts prepared bv school . 

( 8.79: 1 19.38)' Equipment technical manuals prepared by contractors.' 

( , 6.26: 1775? ) Other (please list): : 

.26. How many instructors are assigned to your average I] 'f:. 

-."classroom or learning . center? . .. . . . . . . . . .(M=1.92).. . ." , :< 

< • (^03) : ; v 
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. >27..; How many, students^are^ assigned to' your average 11 . ' •. . ...7:V V . V *i * 

' -classroom or learning center? v ■<£ ,\ ;*> . . .• . .:' (l^?|.'50; v 50=38^25)-^ : 

.:> /: 2^8* HoW[ f^hy' ^desv ^iewbaclcs^: or« prodtrors are^aWigiied-. • ' • ' C ..v. : - v ■ "» 
4 ' - ; P er room/ajro-ve? *' . i'V .•; .• /yfSi . y;^.; . (M=0.35; SD=0.70) . -'• 

. 29. How many full-time managers and ■administrators are assigned ' - 

„ IJtaJthis-ji^Ese^^ 

,"; V '3Q*. What percent of the instructor's day is spent in. ••'•v-. .' 

; %; t draining of ; igener&l military s^ub^ects jpr discipline? . .. ( (M=3.89; SJ)=^Q2.) 

. 31. , WJiat percent of this course ^is- "self-paced",? * ... . ;^ -.^t.(M=57^6l ; SD=45.l3) 

' ' 32.^ What p^rcfenl^ erf %#zi^&&?'&^i-siiep* instruction?^- (M=4 2. 87; SD=45.51) - 

" :' '" vv .- ' '\-"v.\ vlv''.r -^v'V • ■' ••'•»'"•-;•;'.*- - ■ ■: . : '■ / \'f 

• t 33. Do you use ASVAB score^lbr GCT "in. order to get a; ■ ., ... 

1 • predicted' completion /time foj" your students ihDbtii-s ' 

. ^ course? : ;•. . .., ... . . ... . . . . . ... . yes' no 

34-. Are students assigned to special, course materials 

^/ 4 v•^•^.;•-.-•^- %bM |^*••^'J5ql^ t s9 < rt o^ : ;iMtivtude-::S.core?'.' ! '. '. .' . .' . . ". . . . yes no 

> "' •<■' . V*V. .•''•V'v , ' . ,: ( 3 .59) ■ V* '"" 

■ 35.,- What is the average number of working days holding time > '.'■>■ ' 

■ -r- for students awaiting instruction in this- -course? ••^'^ •*< 'v-^M-l.-lS;'- •S0«»&V'i ' 
:■• . •> „ -• •' ' /'.jv (46 Omit) 

36. Estimate the percent: ^of working time' spent -in..each 'of the 
fpljlowing, .acttytM^f.Qr personnel, .awaltlng'.^j'str'ubtion (must •• • ' ' . ;> 

• add to 100): * ' - v , - , ' ' ' v 

#.f. ■ ,|< ' - General preparatory skills, such as^math or reading. - 
, v , ■ Course-related activity. 
) | Military training. 

• Guard, mess duty, etc. . ' '". " 

■^:.v ■ ■ Other Jplease" -list); • 

37. How do students here 1 earn, how well Or poorly they are doing?' 

' ; .^(.fMechanlcally, fron.,tc^^ M - n=25) : : / • • : v " 

,,; ("From'a personal instructor" ' n=10) , ,. . ; ;' \ ' ' ■■(■•■'' 

•?"v/'s. * ("From any instructor" ?'.'«' ? r -.;VPL*f'7)' < r l- /,r 'j .> ' • ■/• ] 

^' ; e--'V ( Combination of Above 'v-.^15 : ;; '^ v '' . * 

•.-'.. •"-■ . . . . ' ':'.•'•''>'.• '••/OV-' ■■. ' » . ,.. . 



. . . . • 

r.'.V '< 



38. List in order of importance the mao'o'r" incentives students have for doing .well 
this course.': J'" $■ ' \''- >, -.'/#'"f- /v' 

•-.-I.. ' ( Extri ns i c.i ncenti' ves ' -7^^-:, n=39) , ' ■ — ... 

•( IntHnis|<$'1ncent'iv$s^ -«ii?S-^j^%«20)^ » - ; - . ; " 
(Combina1;ion of the two ' ' n= 1) 1 ' "••»• .'.-.■■" , 

' v "'-' ■ •'• 5 ■ ;■ ¥ ■ ;:v: '-\^*--\ ; • 

39. List actions' ..available here for dealing with unmotivated students.^ 

(^ "punishment-type" response n=18) 

("Counsel and Help" n =21) : ' \\ 

■> ' ("Special study group" .nfl}:,- . ■•• . • : ■ " ;V " 

. , (Comb.i nation of. above . . . " , . ^.H=i'7p' ; \ V . 

ERIC ;. f /, / : • ; . 43*.; ' 
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4,0; ' List jn , the -order 6f imiportance the 1 'incentives an instructor hcis- for 
. "teaching well here: • , . 1 ; ; ' ■ , t -e^V 

. '! (txtHnsfc incentives ...14)! .. .: • , , v: ;^ ' •?«>■; 

/ {Intrinsic Ir^entiyes' w ^- r V" v . 

41. , Does every % studefft have one instructor '• wHo^?fs 



responsible ;f or his or her. total- succ^ssl^W^th^ 
'training pnbgr ani? .i^ .^^ 4 . ^ . yes .no ■ 

-^^V^r.^ v . ;> ;v , "V..\ ./ . , ^: ; 5v-^r v (25 34) 

42; Which' one} -6^/ the following ideas best' describes . this course? 

" (29) ^ThiSj^iW^ : 'for learning is pretty well, fixed, with 
T * l; • students 'Varying in the degree of their mastery- of 
- 9 the WuKse- goals.; ' « 

(33) . The students vary in the working days to finish the course, 
but the level of their^astery $s pretty much the^aJrie. 



>yj&$> Which one v af the following be^T describes the distribution of 

student achievement in this course? - ;v i 

- i ■ .- * 

(34) A few students below average, 

- above average. ;: * ..r 

(^4);- More "aboVe average ..than below . average: stud en 

v - >& ./ -y-'- m*. . ■ ^ 

(-4) More below average than above Average students. , . 

'44. 7 .Check "the one statement that.isi closest to the philosophy of 
' ;V;v instructors concerning student- achievement here: 

( 9) If we do a good job in giving the' students what they peed * 
_ . there w.j ))• always be 10-30* percent of the students whd just 

cannot .get' -a solid grasp of the subject. 
'"' * " v v: . • ' * ' ■ ■ '• 

,? * If Iwfe' do a good job in giving the students what they need* 
•We can get almost all students to learn almost anything this 
course has to teach. 4 «,.J.. r . ^ 

45. • . Which one of the following besif^d^'ci'vi be$ ii vth.e..- paclhgV in- th i srcourse?, 



All; students progress through the, subject-matter, afVthe 
• same speed— that of an "average 11 student ^ ; V' ^: '?Z:/ v *'* 

..(21)., Students pretty much detenTiirte their own' .individual 
. ^ pace \iri r learning here." ' . 

-.v"' " >■ . 

f24^__TJieJnsii^ 

pace determined by estimates of student abilities and 
the difficulty of . the subject matter. 
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46* Whigti one of , the following best, describes a typical T 
i^tjruoto^ in; this course - ? vV- r : ^ :v 

' (38) y Ttiey ^90 to r the student an\l ask questions /individual Ty* : 
" " " when the student, appears to be v "stumped." . } > 



*<"■ 



: . (23) They. wait" for students to request the help they 



47. 



^ may need. 



w ETIN$T 3,920. IB, "Pol icy orf^ Automation, ■ endorsed automation of .all.' 
education and training functions. Here, we- would; Tike ,t6 know how 
thi s., course Iras usj#> small vmicrocomputers in th j s regard. ^tpp^"J : ^ 
you 'do riot use microcomputers.) ^ • ,' ? 



Instructional Use 



Model: : or, Type?: . NLnititVr vJ' ; 



-Management 



; > • " (Microcomputers) • (6) 



Delivery 



(1) ' ^ 



^Testing 



(3) 



Administration v 



(3) 



Other 



(3) 



48. 



49. 



How many of- the permanent personnel in this course are owners of 
personal micra.qornpiiters? 0 (Number M=3.43 Type Mode = Microcomputers) 
"iv (7 Gdurses) . - 4 



Please fffflicat'e the number of -personnel .'-teaching or supervising this 
course who can write programs in such microcomputer languages as BASIC 
or Pascal . ; . " ^ : .. - 

Introduptory^ Int^medlSaii^ Advanced' '-'^'^i 



(M=4.33 
(9 Courses 



M=2.50 
8 Courses 



50. 



Number at 
eacfi' level of • 
programing, 

Please lHst ; any special uses of microcomputers in thi S' course that 
jni^Wt be useful in v other Navy courses: . ■ ■>. 



6 Courses ) < ; ' ,' 



,,_» I. V 4 



Thank ybii 
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; ■ ; # .APPENDIX - !' 

STRUCTURED INTERVIEW FOR , • [ 
EVALUATING INSTRUCTIONAL MATERIALS 



^r-iThe percent "yes" responses among the 7 CI, 
"C* ,10 MIX, and 20 SP courses are in parentheses 
;N in the left margin as follows; <\ v ;i -f- 

- ; , (CI-MIX-SP) 




Si 
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STRUCTURED INTERVIEW FOR EVALUATING 
• . ' . : V INSTRUCTIONAL MATERIAL'S- : . ' 

ft ••#,/,. -v.- -;:•>'.::•:;•;'•■; ' •••••••• ■ ... •.. 

i. prerequisites, t'^-r- '''"^^-v 

-^'i^matedL^ 



(0-20-50). . Q. 0 **1 . _ -"-Pretest • td determine if prerequisites are -known.- 1 ... 
.(144 0-20) 0* .0,1. 2. Pretest to determine if. student already/ has mastered 



-»-;. M • : lesson material's. 



(29-40-45 ) 0 ; 0 3, Prerequisites „f or a lesson are taught An previous lessons'. 

(14*40-55) 0 . 0 4. Alternate forms of material exist- to accommodate t :v 
.. • :. variations in student aptitude.. • y -[ ; ,. 

(14-30-55) 0 ;•• 0 .5. Remedial material prescribed for missed, test questions in 
'*•;« ' " ■ pretest (for prerequisite* knowledge), progress-checks,' and" 

.'V , . ' formal tests. .. \ • • • > .'/ ' 

(0-40-63) , 0 0" 6; Material is comprehensible to the- typical - ; '\. 

" ''t&y -'f'- student;' i.e. , it- passes the comprehensibiHty checfc.''-' •••'»• 

(0-10-30) 0 0 7. Material shows how.. lesson is related to prior.. learning 

"(advanced organizing). . • " ; ^ . 

.(88r70^65) . :p;; 0^:": 8. K Material states how stadent will." use thisvinformation on - 
- Vir-^Kv'-. the job, to ensure- the student knows why he is- studying. 

-> ii. cues . ''i'-- ^ ' " '•' 

Management documents help 

(71-90-80) 0 0 9. Terminal and Enabling Objectives for the lesson, are- provided 

" to the student,. • . ... ' " 

(43-30-10) 0 ? O 10. The student copy of the. course, outline clearly shows -t* 

Wessons and how- they relate : to each other. 



■ * .■■ t ■ 



j (43-40-65); 0' 0 .sll. The student is given a clear description of how the course 

will be conducted. * I- . 

, Management statements in 'lesson give clear directions *on how to . study the 

' materials. / : > • . ■ " • ; ■■■ ^ ■ ' •' • 

.. : ( 57-^0-80) 0/ . 0 12: Overview describes what lesson is about. <f? - 

; •: (29.-80-85) 0 O ,13. 'Each les^jbn presents, clear d Trect ions to th^-. student pn • 
>> ■• ' . .. how to use the. material. ' •.; . ... ' ' 

' :•' '. ••••• , . y ,. . :y • " ; • ' 

(29^40-60) O Q 14. The student has a. means to record his progress. 
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g Training objectives are, clear and technical ly complete, 

788-60.851 0 . 0 v v 15.,- The describe wh>t the student is expected to N be able to do', 
A .' and give? conditions 1 -and standards. . . , 

/"''....' Information is organized arfd formatted for ease of use. , - .'..V . A :.^:;v; 
Knowledge Objectives:. >...?". >. ; : * . - 1 • ' . 

'r • * N; >. . } . . V ...V'- V V V""' ;.. V.--. v> . ... 

: (43-70-90) 0- 0 i6." All information needed for 'objective, is concisely stated;. 

•••••< in' one place. v. .-' ' •'• 

(57-60-85), 0 ••'<)•/£* 17. • Al T- ''fiot^^s ^Wntt ai -*1 h^d^ayixf^v^s: ecc1ude<}; . " 

(43-70-70)- ' 0" ? 7*18. - Information is displayed'^.blocfts.' 
•'■•9?:. •, • Blocks have names or he#i.flfls.' . 

' V«i7 RO 70V - 0 0 • 19. Verbal information is presented with words; visual 
13/ -our/ y . V. information is presented with graphics. • \ y 

. Performance Objectives: ■ , . ■ •, ■ ; ' \, . 



' 1.' 



(43-70-40) 


0 


0. 


. 20. 


Steps and describe in 'order. » 


(0-10-30) 


0 


0" 


21. 


Rules are pres en.ted to f gui de performance i r> 


(29-30-35) 


0 


,0 


22. 


Explicit safety precautions are presented. 


. .' (-71-90-25) 


0 


0 


23. 


Demonstrations are given. '>/. " ; .- ' ' 


m ,v ( 43-50^-15) 


o- 


0 


24.. 


Demonstrations cover range of applications. _ • 


^ (43-80-20) 


o: 


•■o ;.; 


25 . . 


Medium for demonstration allows students to^stop, start, 
repeat,' and skip forward and backward. 




Memory aiding presentation techniques are used. • ' ' ■. , . : •, !> ^.- 
(0-10-10) 0 . 0 26.* Mnemonics. . ' ' ' , ■ ' 

(0-0-20) 0 , • 0 /' 27. / Chunking. - . ' ' : r . • # ' ... . 

. (0*10-30)' 0 o" ? ; 28. Memorable graphics.- ' ?; v 

" (14-20-40) 0 .0 --29. . Emboldened- key words. ...»_.<•.• " 
' l;. *. 1 1 J. PARTICIPATION -y?- ' : r : 7^ : \^\ ' V ' ; 

- (57-70-75), 0 ' 0 30/' y Students are directed to practice. , ; 
.(-29-80^80) 0 ' 0 ; . : 31. Materials are provided for : |^Wt1ce t :C-y-;.%--J - ... 

0 ;0 '32. Only information found in lesson presentation is needed in 
. (14-80-75) . . .• • practjce. ' , .• ' . • . . v ■ " ■ , 
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(57-80-75)^0 pV . 33. Practice y p^obleifii^re consistent with objective, test . . ^ ; 
v v • - •'. litems and Jessonypresent^tidm - /•' /:-:■ 

—(-43-70^ — -— - 
:■■■/• ! r , U P snd practiced in a series ■ qf^; subsequent lessons* ■'■}$: . 

'(29-40-35) i, p<\ ,35V Materials are pteided for distributed;pr^tic:^ 

(>43-8T>r65) a 0 36.v PwactiVce exqrdises provide knowledge bf results^) , . ■';<■■*■ 

(29-70-45) 0 —0 37. Practice, exercises guide remediation.' : W : ^ \ "vV^ :i v v 

"\ IV 4 . ' RE I.NFpRCEMEN^ ' . . J.^-- -V " ^ .;^V "'v/ : ' -••'..?.:" 1 

(14^20^30) •"; b r ; 0; : ;3j8 • the mat er i al cpntai ns reinforcing Statements to be \ displayed '-- 
:". -V : v . . to : ; the t? studenfr^f ter. tie h as successfully completed a Mock 

' .... \ \\J '■' :■ ■. of instruction^;. \ • - : .. . § ,'. r ;' ". v 

(57-60-60) 0 " -.0 fc 33.'. ;. Students get 75' percerit or more of the; progress check and • 

v . " . ' ttr ':. ■■test items, correct. ■<\ r - : :-l.: ''-'' : '.[ , _ "• ''.My.- . / , 

/ 0 0". v (Mark here if item 35 is checked^ -Tractice exercises . - v : v 
.•./;.: ? . .. ' /prpyide knowledge of resu^ts. ,, " , r - 

V - FE'EpBACK^ 'i V V , - r,' ■" .'V \r ■ - ' ' ; 

: ; .... Progress Cbecks: . . _ V - ./ y 

% (29-70r90): ; ; 0 . -0 40, Written. ' v ; ; 

(29-60-60) 0 p 41. Performance. 

: y • \yC *. Module Progress Tests: . 

(71-&0-90) 0 . Vb, V 42. Written.; , 

(29-90-70) * J 0 \ 0 y 43V. Performance ; 
• - /. Comprehensive within Course Tests: 

(14'rlb-40) 0 ' 0 44. .Wnitten ... : ' 

(14-20-35) 0- 0 ° . 45. • Performance'. 
• r Final Comprehensive, Test: 

' (88-40-65) 0 . - 0 46. Written. ' 

(14-60-30) 0 . 0 :47 ( . Performance'/ " 
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; I Scope of Tests. 

■ i • "* .* 

y N ' - ■ ■ •> ••: .7 • 

•^-t (71-50-85). 0; 0- 48. Test items exist to measure achievement of each. objective.. 



. " - ""' Structure of Test Items/Answers. . * V, ? >%^" 

•{.^i -80-85) 0 0 A9. Test '.items, are consistent with learning objectives. .-. . ^-v^" 

^21^.0^7-0)^-0^- 0 _5Q.:i^Cbn:ect^^ 



•■ (NO data) 0 , 0 ■'■ 51. Test items pass the comprehensibility check. V. v- ;|i 



VI. CORRECTIVES ... 



A: 



(43-50-60) Q ,0 * 52.; ..Directions ba how/ to. study material on missed Items are ;' ^ 

■••■■."■"<■ ' provided. * . ' ' ■ " ■ 4 - v. " ; • . <::;'<%^v^ -J 

• : . :{^-80^55) : \0£ Q " ^ (Mark .Vere if item 36 is checked: "Practice exercises guide - ■'- 

" remediation..") . ' >■ y • 



(71-80-75) 0 : . (.Mark herj^-i.tenr .49 is checked: 7 "Correct answers to 

" i ^missed ite^^re- presented to students. 11 ) * ' ■ ■ 



f 

I- 
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"> COMPREHENS IB ILITY CHECK 



V 



v A:, i . ^ From each set of instructional materials, randomly^.selett .3 narrative* 
. passaged of about 500 words each, and conduct a. CRES analysis on these* passages! '£ 

• 0 C 0 Less than .05% awkward or undesirable words;. ;. : ; 

0; 0 Less than 10% long sentences. " . V.--«r-:" s 

0 - 0 No "lists in long sentences. - 

-^v'. ' 0 . 0 / ;Not higher than 10th reading; grade level (12th for highly; 
; technical material). y m ... 

B. * fr^ materials, randomly 

select 20 questions flO m^lUple choice; 10 T/F,';if possible) and analyze them. :■ 
* using a modified CRES routtrt^ . ' .'4 * : . ; -\ 

-/>>,-- l. v In- multiple chofte questions :^ ; ; ? * : : / ,' 



Mot more than 25% of the' correct answers are the longest answers. 

No questions flagged as having inappropriate answers. * - 

No riepetitive words and phrases in answers. 

Double negatives rare not used. ^ V ? . ~ 

Not more than 15% of test i tern al ternat i ves are more J thar^ . 22' : >v 0 v 
words. ' " • . - ml 'v 

Reading^grade level of test items does ijpt exceed tlje 10th grade i 
(12th for highly technical material). . 

2. In true/false questions: 

0 0 Not more than 15% of questions are more than \22 v WQ»^vi\'- * 
0 0 No negative wording in questions. . 

0 0 No complex questions (i.e., "either^ .or; 11 "neither... nor, 41 and — 

. ' . . - ^ "or"). : ; .- . -\ : . • 

0 0 Reading grade level of test items does not exceed the 10th > grade 
. " (12th higRly technical material). 



Y 


N 


-cr 


0 


0 


0 


0 


•Q 


0, 


0 


0 


0 

1"; 


0 '■'} 


o. 
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.QUALITY OF INSTRUCTION QUESf I0NNAIR€ 



Data- are recorded from 37 courses in final sample: 

Group 1: 7 Conventional . instruction. courses (CI) n=146 
v Group 2: 10 Mixed 4 5 - 90%) Courses (MIX) n=210 - , 
■' .^^jGroup 3: 20 Self-Paced (-SP) n=958 . ' v^} ' 

' '' Questionnaires scored l,.to. 9. Group mean is indicated on the 
scale with alpha probabilities'' from oneway AN OVA. Humbert total = 1,314 
cases from 37 courses at 9 locations. ' . ' / 



9 
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This questionnaire was designed , by the Training ^Analysis and Evaluation 
Group to provide a snapshot" of feel ings concerning current instruction. 
Your responses to the questions will become part of a pool of responses and> 
will not be traceable to you after they are entered, into the computer. No ' 
student, instructor, or supervisor decisions. related to reward or promotion 
may be made on tjfe basis of your answers here. - « 



Name 6f Course Being Rated v 
Locatio n " ' 

Check one: Student, . m Instructor .Supervisor .* 

::. v.,,;.: . ■ -i'.'-v ...... . ... 

vHpw mahy;:16ssqns (modules) in this course? V :v 
For students: How many j^^^ 



DIRECTIONS: Draw' a small vertical line through the point on the two- ' 

sided scale that best summarizes your answer to the question.^ 
For example, the response to the question: ".•;.'!,•. 



1 



... . "How : do you. feel today?" " \ . ^ 

Bad ! \ - — fe, ■ . J ^ ! Good 

/conveys the notion that you feel pretty good The 'remaining 
questions are 'harder. Please try to answer every question/ 
•' The. summarized information from hundreds of these questidn-, * 
naires'wijn be^a gujde for: the improvement of Navy / . 
instructToru -' : fh\ ]--.\ , ;>;...• ' 

— — - — ■ . V-fe. ; ■ ; ■ ■ - ; \ - •■- 
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1. . How often do you find course materials here to* be written as if 

students have Certain ski lis or knowledge that they, in fact; do not 
have? •'■ ■ : - - ,: \ . 

■V?# CI-MLX-SP . 

J Often. 



CI-M 
I — --■ 


jx-sp - 






■ i ... . . . , .. ..•■•>■: ■• . ^ _ 



H 2. For most students, the reading level of lesson materials in this ■ 
course is * » 

m CI-SP-MIX 

f Difficult I— |[ . | 1 Easy. 

3^r— To-what-extent~-ns~rev^ 
course? 

,r Much i sp -f- ci jLiuio. 

. 4. Prior to enrolling in this,, course, most students' attitude about 

school learning was > - . ' . ' 

MIX-SP-CI 

■4 Good. 





MIX-S 


3 -CI 


1 _ 




— — — \ 



5. BasM o;6 my know edge of the average abilities of studerits here, I 
fim J^ter them as v 

ci-nix-sp 

■I Underachievers. 





, CI- 


wx-sp 




1 — 






— -i 



6. Students 1 study habits are improved by this course 



; ■ , MX-CI-SP . 

""• " Much I— — - M — — J Little. : 

• . , - . »' -, . '.vV'-.; 

7. Before students can take an ^examination here,*to what degree must they 
convince their instructor that they are ready for it? 



Little t— r ! -Hi : 1 Much . 

8. How well is "tb^'nformation to be learned in this course presented in 
: the study materials?, 



■ -SP-CIt 

Poor 1 



MIX ' 

— 1 Good . 4 



9. ; In this course the learning objectives, seem very ^ 

' ci-mXrSP ' ; v 

Specific J » 111' — — I Vague. 

10. To what degree does an instructor assist in getting and holding the 
student's attention to the instructional material^? • 7 » 

. - ^High 1 .. . :[\ , ;:j4:fl;r::, ; . J Low. - ; • ^ 
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11.. To what degree are the instructors in this course expert in the 
subject? : ..-.« : . . \ 



, . sp-pix i. 



V : •' 



12. The course materials help students know where they' are^ in the course 
and what they are to do next. * 

• • • ■■■■-m '.. ; : vMI.X-SP-CI . • :. 

.: Agree 1 [ fl 1 Disagree. 



— 1-3t— What-proportion-o^^ 

individual students, in this course? 

SP-CI- 



-CI -MM • 

All of it" I— f| — 1 None of it. ..//'■: 

■ r 14. .What, proportion of the instructor' s diy-M: seemed tb: be spent gtv-tiuj 
directions or information to students'? - • ' . 9 

• ' ' ' • ■" MIX-SP-CI & 

■ ■ ; L1tt1e ' . . | J | ... i Much. 

,15. It seems as if most of the information in this course. comes bv 

Printed materials,, ! T —! Lecture. 

.- or demonstration 1 1 

.16. Compared to other courses , the amount of time between presentation of 
a lesson and the practice of its skills here seems ' T 

: '; ■■ • CI T MIX-SP : ' • : 

' ... Lon 9 1 . Hi i Short. 

17. The lesson materials clearly describe why it is important that * " 
students learn the information presented in the lesson. 



Agree H 



-I Disagree* 



18 • " In this; bourse what amount of the student's day is spent in 
; constructive use of time in course%activity? 

LitUe ' > y . [[] . I . Much. • 

19. To' what degree is the wait4g t.ime between arr.ival on base and the 
start of this course used th course-preparatory activity? » 

Hinii t MIX-QI-SP ■■ ' 
Hl 9 h 1 ■ ,, ■ fl 1 LOW. 

20. How much study time do students usually spenii beyond the scheduled 
-:. academic workday? 

2 '.* The course lesson materiais here Jib1«d students' ihte^st for • 
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22. r To what degree do the teachi^hg-Uiethods used in this course seem 

appropriate? .. 

■ CI-MIXrSP 

;-:-'-rH1gh I I j[ , -r-?i Low. 

23. How; easy is it for a student to daydream in th>i$. course? , 

• Hard J — — | — -[) r — -i Easy.. 

JA^Aeneraliy^ _t ij ^ _kind_o_f_p LaciJ) gJthkt_n»s.t_s±udents_do^bes-t-under-4s- 

• . ' •.••.•<>;-: V ; H |V]JY_5p ' ' 

: • Instructor-pacing J , f , . ]| 1 Self-pacing^, r, 

25. to what degree can students- here progress at thei r, own rate? ' 

' • : ' , ' CI \ 'Ml-SP . : . r. . 

. Low I . .... [ ] -J. High. ,. 



.^26. To. what degree dpesvan instructor determine the rate students progress 
''{■■■'' through this course? : 

,> • sp-jwx c: 

Low- j. — : — f- 



-4 High. 

27. Compared to other courses, how/much .time do' students here spend • 
practicing the things they 'have: 'Tear" 

MIX-SP r CI . 
Less : J— ■ [| J , . . ; / 1 More. 

28. To what degree do the practice and lab activities of this course seem 
like wbrk° you may someday have to do? ; 

, CI-MIXrSP. 



High I . , . 1 Low. 



29. Practice activities will >help students to remember course material 
when they get to the job. '>- ... 

CI-MX-SP 



Agree I | f| — — - ;. = . — ! Disagree. 

30. The lesson materials proyide'studetits here.with the chance to practice 
using what they have learned - •* " • 

*• ■ v --.••v^ - mx-.ci-sp •• 

. •' Seldom,: l— ' , , H \ Often... : V*. 



31. How enthusiastic are the instructors about the subject of this course? 

CI-FIIX-SP 

Much V j — \\ ; — —r- — ; 1 Little. 

■32. To What degree do the instructors, seem to be recognized^ by their 

, /supervisors because they give a lot of individual help to students?;; vfe 



High. J- 



. . Mi 









■rl LOW. ' 



,33. ^Instructors here seem. to believe that jHT students can learn the 
-subject martter. 
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, 34. .To what "degree do the, instructors here work to ensure that aTV ;: 
• students do, in fact, learn? -• • :• ^ 

-. High | |f| ? H Low.e V V . 

^•35. Students see the information in this course important for their future, 
' "• •. civilian career. ••••>'. 

Disagree I- ■ — | [ [ , 1 .Agree. 

36. The. incentives for. finishing thjs course as quiplcly as possible seem 
• . -f >i - ' 'MIX-SP.-CI . * Mr . 

;; 6reat I . — Small. ; ' 



37. Graduates see ,th i s course as important to -fheir military career. 

Disagree ! ^ . - 4) ] t Agree,,;- 7^ 



"38. How highly would you expect students v to recommend this course to 
others? " .> ■ • •• 

, .L' , ,. SP-Mll-CI?" • •* 

Little I — 1 1 Highly. 

■Jt 39 - a 'Wheh'- v ittideh'ts^.o poorly. -on a test, to what degree- is correcWe^ork 
all owed? . . ! 

, . " MIX-SP-Cfc. .-, i 

: Great »- f — —i Small., 

40. The amount of testing-. in this course seems to be too 

' - SP-MIX-CI 

*■: MuCh ' ~ (f~ — ■ H Little. 

• 41. How often can students retake an examination; should they f ail?,: 



•1 bLuuents retake an examination; should they fail 7 / 

Never 1 ! — ] 1 J 1 Many times. 

42. Testing in this course occurs almost • 

V-i ,SP-fHX,-CI ' ;\ 

Daily J — . 1 Weekly.' 

... 4 .^ X! t ? S J S 5 e r e u really measure how well students know what the lesson > 
, • materials teach. ^ 

• • theTt^ e rtest? eS instructor become inv °lved with students"; after . ' 

i,v. ' - r -%-W X-ci, * . ■ >• '• . 

• ^ . Low. »; 1 . I) [ — -I High. 

45 V %eems^o,me that the amount of time allotted for re learning after ^ 
•V V Wwre, A to. pass an examination '.is too • 

•', . .. Little I— , ■ ■ ; . I Much. 

ERIC" ' ' V" ■ " ! . ' ,.: > ' ;\ . . : :: . . : .. ' ; . -;^ r ^^ •;' 
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46. -Compared to. other courses, civilian or military, howmucJTdo the 
, instructors here seem to be available to help students? 

" . sp-cmix; 



Little > 



^ Much. 



47. To What degree do students take individual corrective activity in this 
classropm, if they should fail an^important examination? 

MI-X-SPrCF 



High t- 



4 Low. 



48. -In this course, students who fall behind are set back to a later 
■ J ■ . class. ' : V:-."' ' 

■ , sp-mix 'ci ■ • • 

• Disagree 1 | 1 ] ' . — -A Agree. 



49. When students miss terms in practice exercises and tests, the 
avai labi Tity of the materials for restudy ' is 



Good I- 



•MIX- 



-SP 



T 



Poor. 



50. Following failure <Jf an examination, how many different ways of 
learning the material are. there available? 

. . : . ci-mx-sp . . • • 

None J — i ■ f I Many. ... 



Thank you. 
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APPENDIX D . 

. ■' MEAN VALUES AND SIGNIFICANCE (Total df- •= 1313) OF 
THREE VARU ON QUALITY OF INSTRUCTION QUESTIONNAIRE : 

:■' (These are the numerical values for 
: the scaJes plotted in appendix. C.) 
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TABLE D-li.' MEAN VALUES AND SIGNIFICANCE OF THREE 
\ . VARIATIONS OF II' ACROSS 50 QI QUESTIONS 

Question 




*n . s . -not si gni f leant 
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TABLE 0-1. MEAN VALUES AND SIGNIFICANCE OF THREE* 

VARIABLES II ACROSS 50 Ql QUESTIONS (continued)- 



? Question 


CI • 


- MIX 


SP 


ANOVA p 


44 


6.49 


5.49 

— 5^03— 


5.47 
4.59 


t .-.01. - ' 
' .01. • 


45 

•:■ 46 

>v.;-.":v;' • • 48 '■• . 
■(&'•>/'.• ' 49 
«-*.'r-'50. 


4.60 

7.35 
6.12 , 
4.23 
6.95 
- 5.92 


' 7.38 
6.77 
6.03 
7.36. 
5.92 


6.37 
6.54 
7.28 
6.84 
5.95 


.01 ' 

'■- ■■ 105 ' . ■ 

• - .oi^ ■>**••. ■ 
: .01 

n.s. 
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APPENDIX 2 ^ 
REVIEW OF THE LITERATURE 
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REVIEW OF THE LITERATURE 

.• » - ! - ' r ..- • ' ■ ■ 

HISTORICAL: , ' ; s , 

. The conversion of Navy instruction to • the- individuaTized learning 
center,, activity ^that is, the subject of this report ; hai antecedents in the 
* beginning of recorded history. Brubacher 1 s* A History of the Problems o f 
. Education (1947) g i ves_ -a number of prec edents~for modern trauii ng meth ods, t 
among them': .the sorting of students who were to betome the philosophers, 
warriors, .or • artisans based on individual differences in mental talent; 
described by Plato; 'the advocacy of different teaching methods for differing 
student natures by . QuintilTian; and : more .recently, the 1919 . Dal ton, 
Massachusetts," plan allowing students to" progress at their own rate, 'and 
; Carleton Washburn^s in II 1 incfis also, in 1919. 0 

: ? ' The efficiencies sought today by accounting for individual human 
^•■■: ; "^ifferehces...in learning ^were foreseen by Terman oyer $0 years agoi 

If the differences are found due iri'the.main "to.-v \ y , 
. $y, v : /control 1 able factors of enyirehm.ent and training,: then, r ; . 

. -i • theptetic^ out by. appropriate • 'X " . 

education prbpeduresr^procedyre^ would- theh be/' * -%s» 

obr duty to provide! On the/ othen hand, if they are ^ ^ 

' / i. primarily due, t^ 

v the duty of the schodt is; cl<^r]^ -*^v. .. 

^ ^di'fferentiated tr-aipin^ ; . 

: differences into accoimf^erman^ 

THEORETICAL - ; • : . - 

■■^^;- v \vTh]fs- was the thought that was particularly attractive to trainers in 
the ^ 1960s: who found the need' to teach a > widely divergent group , of 
individuals to successfully pursue highly cohven^ht learning objectives. 
' Prior ' to this time" researchers dealt wi.th individual differences as the . 
within-croup variance in their experiments.. Jhis variance was the "error 

• term" i I their F and jt. ratios; Good experimental design sought to maximize 
the group .Vari aFiori under f - study, control tie extraneous sources^ of vari ance, 
arid, minimize -.ttfe* error .variance (due to individual differences in the 
subjfect^ or //'learners) (Kerlinger, 1973) ^ ^Sut^ . sucl^experirrientation 
confounded the differential effects of » individual d if ferenoies. Strategies 
of schooling basefl on this research made matters worse by emphasizing the . 
very differences that were such . a problem for the group-based .methods in ; 

' - us e.: .■ "i ' : ■ V; > "•• , v.'- 

: ' "' ' v , ." ' • "' v • ' ' ' ' < \ -V-.- .'. 
The untangling of the relationships between methods of .instruction aW- 
. ways, of /learning has more recently oqcurred in investigations of "ab^lity- 

• treatment interactions" (Bracht, 1970; ^Berliner, and Cahen,^ 1973) , ^ 
Theoretically, the more we know^ about how various individuals learn" various 

:^:frinds< d^thi.ngs t;o be learned the better the learning environment can be ' 
^cdbtfdWed*' and the less the "error" * term in a teaching- or learning v 
experiment; The more treatments an instructional . system has available to." 
.teach a given task, the njpre likely ' it will have an efficients an^ ^ff ecti#i^l 



^■ ^ymZ. • .■•■•v-.v v .*-.':* :: ; 

■ *^v!V'^ '.' ' Technical Report .141 = ■ • • ; - - ■'•.,> " ; '../.'.'■*■;■ 

; ; ;: >i;.^ : 7 . • , • •. • • ■ • • : - • • ■ ■ 

^ method to reach an individual of a given- ability, to extend this logic, if 
• we ^had enough ways, of adapting our methods of instruction to a given ability 

student-,- the variation of human achievement should approach zero— that is, 

almost anyone should be able to. learn most anything. 

Fundamentals of such learning were first described by Miller and 
Dol lard (1941) in an eVly theory of teaching. . Their identification of- 
drive,; cues, response,, and reward > was a forerunner to much of the later 
identification of the. behavioral characteristics of teaching— most- 
l^o^antJy^sed^y^ar^P^^ea^'and^then-B-loom-^-ge^ Most-of^-the- 



current efforts in individualized instruction have philosophical roots in 
Carroll's "A Model of School Learning" (1963), which asserts, 

; ' • ' : feree of Learning = ? fe^U 



"TimYTieea r ea r . b *' if* : 



"Time spent" is further differentiated with "time allowed for .learning" 
and "time the "learner is willing to /spend," or perseverance. The "time 
needed" is also broken down to : that required because Of the students 1 
aptitude, "abjl ity . to understand instruction, " and "quality of instruction" 
(01). Thus, fhis model final 1« becomes, 



' '"Degree 'of Learning - ff:^ 

/ t Time req'd^T&^feUnderstancTTcjTV 

The development of. these formulas is expanded ih Block and Burns (19.77) 

Benjamin ^B loom (1968)- reasoned that' by optimum presence of the 
variables under the school 's- control it would be possible foV almost all ' * 
students to attain levels of achievement heretofore attained by only a 1 few. 
The application of thi;s reasoning was' ca-lled "Learning for" Mastery. " This 
notion was well suited for a- military training environment— if under; fixed- ' 
time instruction students varied in the : degree of masteryv : - i? and .if 
achievement variance -around a given criterion <for mastery is undesirable or' \ 
costly, then Bloom's system, fixed the Jevel- of mastery allowing. ; students to 
vary in the time taken to learn. This idea became>a -'philosophical basis for ; 
much of the so-called "self-paced" course development introduced in military 
instruction during the 1970s. .4 , ' ; , • 

Bloom (1974) has further^easoned that, the apparent variation in human 
intell igence, aptitude, and achievement is based on norms - ref lecting the 
elapsed timMindividuals' have lived, or elapsed time . exposed v to school tng, 




learners ;at the beginning of a course, is about five times that of tfe^Sfe!?^'^^ 
■ learners. "■• Furthermore, when yoa eliminate. "the : wasted time and tMmf' 5 - 
''elapse^ time on task, this ratio , is^reduced. to 3 to. 1. Finally,, Bfoom' has "" ^ 
observed that, students nearing completion of a,* course taught by. mastery^ '> I* 
learning, methods differ rbrik about 1 . 5 to U ' 1 • l^.r s : - ' 
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, y r ' Anderson (1976) ' investigated ; the magnitude 'and ' 
: individual differences in, elapsed timi' and time on 
criterion* He found that rate of learning defined 
'alterable, much as* Bloom '(197.§) later hypothesized in 
thj^qry a of ."school learning (see - figure E-l). v Prominent 



Je of learning" 
a^rtion that 
the ratio*of 



stability of */&he 
task and' t ifrre.VtQ 
in both way's p was / 
. a - ful ly , d4vl:tojped. / 
in this nftdd^ M 



as a dependent Variable, in keeping with th^e^ief '■'/. 



STUDENT 



CHARACTERISTICS 



it in a given sUbjecl t§gght with Mastery Learning ■'■^kiieV/' 
TTM* between' "slW.- and "fast" learners* should diminish^ : m) 

" INSTRUCTION - LEARN I NG . ' '^m^ ^'J& : f - 

outcomes .Wm^^M^ 



Cognitive Entry 
s Behaviors* — — 



Affective Entry 
Behaviors — rr 




— ► Yeyel an# 

Ach i evenieri$§ 



♦ 

Quality of 
y ihstigjction 




Rate of J 



Affective Outcomes 



Figure E-l. . Major -Variables, in the Theory of School 
(after Bloom, \976) 



Learning 



ft" 



Taken v £epar^tely,« •* Bloom >sser1^-f "that 
(intelligent^: and experience), account^^br >§0. 



Cognitive- 
percent of . 



. Entry Behaviors 

- . v . v , . ^ - ...... the variation in 1 

school achievement , Affective Entry^Behavici^s-: (aftitucjjK about learning) 
account fof^5 percent," and Qualvty'Vof instruction -acc&unts for 25 percent 
of this variation^; The interactive combination of • -these th^ee major* 
variables accounts r for aboiit 3tf\perceht * of . school achievement. ' •Since, 
cognitive and ''affective -entr^v behaviors' are nearly, inalterable during the 
length of mq^t courses? in tfie .Navy, .we will deaj* primarily with those 
elements that- are. Blpomj^j . tHeory destr-i bes- six el emerif s that are present 
. irt^ quality instruction, were .explained in the .definitions section of ' ' 

this paper as ^prerequisites', cues, participation; reinforcement, feedback, .'< 
and correctives. . . , v ■ ."" '. • .■•/'. ■ 

■ •• *0 -: ■ ' . ■ • %f . ' , : ' . ■•' ::i 

These elements of? quality instruction were discerned by looking* at the-: . 
ideal Reaching '.$0 learning si/tuationr-the tutorial. The. tutorial is 
perhaps the easiest s.ituatiort^to proyTde fofl-the foregoing six elements of 
quality instruction. e *The tasfe'of Navy instructional developers is to try to 



• • • ^'^ V ; ; * ;:'.' -Technical Report 147 • , u ■■•'«, 

design these ideals of the»'one-to-one student/instructor situation into the 
• many- to-one situation. ;"' , -'•-,,.'. ^' • . ' .,••_,' 

QI VARIABLES ' - ' " .•'■:-•""' ■••'■ . 'si^&h' 

i-'' •;. ' -The kind, of individual izing'.that takes pl^l^T.a- given instructional .. 
'•context depends on?' the number"' and kinds of tools that are available in the 
instructor's kit; . . Matlock, \et al '.. ' (i'980>-^bliSd.ucteid::-..ah^xte»is-1ve review of ' - • 

th e literature on tji e 'subject and postulated eight sucW control ling factors: 

(1) time avavT-abi^; (2) instructional personnel , (3) facilities, (4) manage- .. 
, mentv (5) itudent: population characteristic's;- ('6) course" content/task types, 
f%liD.? tructlonal methods, and (8) medi a/materi al s/dev'i ces . From, these, 
^e^P^'o Posed a. four-dimensional model for classifying a v nd describing models 
JfT'Tf.' The baic dimensions were objectives, time,, proficiency, and 
instructional trpitmen.t--each of which could be fixed or variable. ' 
Supposedly, courWs could -then be assigned to one" of the 16 categories of 
"this 2x2x2x2 model for comparison. '.. Salient conclusions from their'! 
^literature review were: 'that' II is not a "precise term, that there are' many 
approaches to II, * and .that there "are . few fundamental differences, among 

- £urrent-:approaches to individual izinq*, - 

. ' A study of the effects of cues" ,*'•• participation, and corrective feedback .. 
<#in instruction looked at . 54 studies involving 700 classes ( Lysakowski ; ' and 
' Walbergi^.1982)!: : The characteristics of over 14 ,.000- students of these 
clfls/es'i suggested large-and consistent effects ■. of instructional' cues,' • 

- participation,' and corrective feedback for. learning in natural ' settings, 
such as. in a typical classroom. The authors concluded that their' analysis 

- strongly confirms the Do l^d-Miller-Carro.ll -Bloom- theory that has evolved 
during the past four -decades. • :i > ^ V - ^ ■"' 

Several . studies deal with the, instructor/student (I/S) ratio 'and its 
, effect on achievement. -Glass" and- Smith (1979) conducted a meta-analysis of 
'80 studies of class size with achievement, finding an inverse relationship,* ' 
and concluded that few resources at the command' of educators will reliably 

- prodgce effects of the size found. ' Glaste and Smith suggest, for example, 
that- a student who, might score .at the 63rd 'percentile when taught 
individually, would score at about -the 37th percentile in a class of 40 
students. Van Matre," Vet al. "(1981) compared 237 Navy BE/E students 

• undergoing instruction with a 1/18 I/S ratio and 1/30 I/S ratio.' They fouad 
_that the Jarger class , size may have detrimental effects on student time to 
^mastery, it . als#gmay have an effect 'on, instructor administrative 

^^eha^idr.^WWien, -;KuW, '• and • K'Urlik (1982) conducted a meta-analysis of 65 - 
school: .tutoring programs.. Here, t^e tutored students, scored higher on 
examinations and, had more positive' attitudes -.toward subject matter; Another 
recent study by Kulik and.. Kulik (1982) looked at ability grouping as .a way 
to individualize within a conventional : instructional, environment. This was 

. a raetaf analysis of 52 studies on the subject, finding a smalls effect si ze§ *' 

, bu#^ef#ect Targ§' enoflgh to move a given student from the 50th to -the 54ih 
percenti >e in achievement.. . -'-"*;/• c_J/ 

•The meta-analysis procedure itself was investigated for its rigor in 
making conclusions about class size (Hedges, and .Stock, 1983). Usinq their. . 

•• ' :• •.. '. •.. . v , • * ■ • ■ . . • • *•/ ,m ■ . 




;68 ■ ■ .• 'T-'Wl 
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£ own model of efffec.t size, in. reanalysis, however, did not suggest any 
substantial changes in the conclusions of Glass and. Smith. The authors' 
concluded, that there is higher achievement e^pec'tatiorKin smaller classes 
and that class size accounts for substantial ^aiSunts of variation in student 
achievement. • :• ;''•-.*. ?• . ••• • •' * • . , . . . 



Morris- Surber, : and Bi^otf: .^$1978) -^estigated the", effect'/; of 
procrastination in* 75 • college students .in an "individualized course w'ith 74 
in the equivalent of an IMI course. ': v f wlpd- ' that'-'evjeh'- thQugh £he"i- "self- 
paced" students procrastinated in .^^ours^acti'vntes^-whiTe those - in the 
instructor-paced course did not, both:- groups scored as'" we 11 in achievement, 
retention, and attitude measures. They concluded that although students 
r procrastinate when free to do so, they proceed evenly " through the course 
' ' material when given incentives to do so. Finally, - the authors remind us , 
that ".^...students do not self-pace; they pace according -toithe conditions 
> that cOntro|p|icing' behavior" (p. 228) J • : ' / ^^/ " ; ^ 

/ A correlatiopal styfyf><yf 385 freshman biology stxiden$f^^d^we^e under 
• .individualized 'conventional instruction 'fouhtf^ci&^I-ndividual , 

differences in liners important. Latta, Do! ph i n , and "Gr^be "(1978) found 
Tow-ability, high -test-anxious students do best under. II— especially 
'* -females. ^They also, found perseverance,, measured by extra .lecture attendance 
and extra reported study hours, ppsitivel$trel'ated to performance in*mas.tery„ 
learning strategies; but in .traditional y •instruct ion, this was. -only true, for'* 
males. x .•' . 4 ; '/ " 

^ "• ' -x ' .' " • *. 

Time is. central to the- Carroll mode£%; Centra and Potter (19)80) 
developed a model depicting their notion of the variables contributing most 
to student;, ^earning. One of • their variables, Time on -Task, was often cited 
^ in their review, of the literature. They note that it may be overly 
^simplistic i:o conclude that .achievement increases merely dup tp - length of 
• time in school, as niiany researchers have found. ' But 'Centra and Rotter warn 
v that * this is not" enough--concern must -also exist for the quality of time ' 
spent in the j^rnijiq (ejivironment. Walberg, (1982) - synthesized, three*. 
National studies of educational effectiveness. Although many studies •show" 



.' . The importance of the effect of -fflfc ratio found by tfan Matre, et*al. 
(1981), us- supported by \ 'more re^fpNPRDC study. Johnson" and Graham 

^982)-kept^ec6rds~of-BEyE^nd^^^^gstructors-in-the- CMI~ eWi"ro"nme~nt~t6~ 
determine the kinds of activit^^Pent, initiating factors, 'and time "f" 
devoted to each episode of IJ^^Mfvity.- They found that while jobs 
differed^ .greatly ,betwe,en -and • wySH^^dpursesw .the demands on the learning 
center- instructors, depended .on the : Te^hgth/ a?id 'difficuTty of the learning 
module .and that tradeoffs between training effectiveness and; the demands oh ; 
the instructor have not always been tnade. » The mo^L striking observation was «.•."•• 

; that in these comp|terizefi courses most i' ristr^U^ris- ^sgejid their . time in ' 

Routine -trans actions 1 - with the -student. , • • 'W 

In a survey of 255 students' and. . 100- instr^ 
San Diego and Memph is , , Rob.1 ji son ,- et al . --(£981-) ^P§&nd ~- tfiat trainees' 
attitudes were generMly ^ instructors' 
•attitudes were riot favorab'lef. They also rioted that thei longer ' the- trainees * * 
stayedj'h the Navy, r ,the more i:hei> ^ - " 



■^Technical -Report- ■l47"./. : . : . : i ' -^7 '.V ; - . — v ^'/ . : 

no. significant difference between groups; a "Box Scored of a percentage of 
studies favorable to the relationship between time a*d learning is 95 
percent positive. To make hi£ point,; Wal berg .suggests that doubling the 
time* students actual ly co.nclfftrate^ on study r might double the .amounts of 

learning. -'-/v .'.7 . ,. v .', ■' [ 

Block and Tierney> (1974) compared two corrective procedures in -a 
Mastery 1 pgrning instructional strategy: ■ correction by redirected study and 



correction in the Bloom meth'od, which was high in feedback and formative 
evaluation. v The redirected : study correction procedure had 130 effect on the 
kriowl edge outcomes of the upper rd^yision historiography course, but the 
Bloom-type increased the appl^ n ' ^ ¥ : 

A< -recent^evt ew 7 . of ,U -s^ es sponsored v by • the Army Research Institute 
was concerned with the sretent ion J^k^-aerf ormed wi th i ri the operati^aV 
military envifonrifeht (Hagm-an and. Rose; -1983|^ v *It concluded that retencW 
was-- improved' by repet i tiQn^Skd^^ \ aoc||^aining tailored ¥or k 

specific environments. It 

in the use of equipment variety'fi^ and 
repetitions are. spaced, and tha1$il^ ly 
useful.' 7 - ; " 

Thompson (1980) compared anMn^|j|^^ ' teaching 

' calculus .with a conventional 1 ec^re-|£^^ .among 
840 Air Force Academy fre$hmen. "w in math 

achievement, and suggests that difference^ ^ 

be due ,to the inferiority of CI, . rather than" the superiority of II, since 
the CI courses in his study were thought to* be well -taught. Thompson 
' \ • suggests '"Wat a, less-professional instructor is^n^eded in II, Which makes it 
nmore * cost-effective. * -ll^also warns that assignment of CI^cfitKlitibned 
faculty to IT courses rn^y result; in a" ''1qss"'jjf job^ satisfaction anting* 
v instructors. ^ ^^W^ „!/-'-- ; - v "-\ " 'C^r 

.♦<,.'' costs ;■. / v . . \ 

TAEG~Technical Report 105- contrasted the operating: costs 4 of - CMI, tlMI,'*: 
and CI in a Navy RM A course "(torey", 1981) , showfrfg nearly : similar ahntolv 
operating tosts, but with -15-^ear cyfle/operating casts' favor i-ng ^pJI; and % 
CM-I. -vWhiJe. no "general izations ri: were made- beyo'nd^ the^ course "studied, ttie 
report did coftcli/de' that the most' expensive- resource ^n t trtining ;is the.* 
Student populations/and that .dollars -spent , in ci^ridulum ^development? show 
s * great potential .for payback *in~ a- short time.- The re J^t/ a l?o ^ecommehded* : 
thSt f QiTnal /economic analysis be"a_part qf gvery^ major o^se-deYelOpment-/- 
W"> " - : %; - - ... .• * r . ' ■/ >, / 

^Orlansky and" String* (1979); studied jpostr^fectiveness 'of *£omputef~ 
v < based instruction in alj^of the military /'services: /They cohcluded* that^ 
' while achievement din - CMI ; , CAI-; and CJ are abotit the- same, that computer- ., 
based education ^ to mastery oyer ■ CI, ;A.r tatec 

study ^OrTansky, ^ ; 
* - simulators*, Qpmputer-based instruction, and maintenance training simulators * 
^rjf [•} ;^f^cos"t and effecti^veness/wi^t-h nearly. the same conclusions. Tfiey also cite .-; 
V summary 'figures ^giving some indication .of the importance of training to -"the 
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Department of Defense— a $12.8 billion enterprise in FY 83. . The iitfie spent 
by students, instructors, and others* in individual training account for 
about one-fourth of the military and civilian man-years 'in DOD. About 20 
/ . percejit of al 1 military personnel are in schools as students or instructors, 
and abou% 76 percent, of this effort is initial- training for new service 



..^ : "';V -members'.- : ~ ,, 



The* various versions '«fllidT't^iita^e ; (^^#uctijOT have antecedents in 
the wisdom; of recorded history, with solid roots in the learning theory of 
this -half century. From the. tutorial to the large group presentation, there 
are definable attributes of effective instruction that are not present in 
ineffective instruction., vtte reconceptualization of schoor organization \ 
possible by accepting learning time as a variable allows the' planning of 
instr ^« 0 -5L that can nold level's of performance constant. This allows the 
deveJlpfent^of a. new philosophy of training that can retain high : exit 
requirements from courses that may -not necfessarily have control over the 
-•ability levels of students entering the programs. \ 

Benjamin Bloom's theory of," school learning^ is seen as the' tacit 
philosophical foundation , of most military individualized instruction. 
Adaptive' instruction accounts for the cognitive and affective entry - 
behaviors „ for • a course by designing instruction Wtr known levels of 
prerequisites. - Instructional quality is ^etermineiF by the presence or M 
absence of cues, participation, reinforcement, '^feedback, "and correctives. ^ 
When ,these six attributes are " optimum * for ' a. given human ability, the 
philosophy guiding -Learning for Mastery can. assert that all can learn, and 
all, will learn* ;V The philosophy asserts that ''it ' may - take- .more time and 
effort' for some than for others—but they mil learn. .. r " 
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A MODEL OF INDIVIDUALIZED'INSTRUCTION 

this appendix attempts to synthesize that which was learned ^rom a 
review of the literature 'and that j^/hich was . obtained from site vis'Tts"' and 
questionnaires. Figure F-l is a fjow diagram describing ideal 
individualized instruction. It is consonant with the instructional theories 
that provide the foundation for mastery learning. It attempts to synthesize 
what was learned by visiting. 37 Navy-land Marine Corps learning centers and 
classrooms. Finally, it assumes that all students who begin the process can 
and wi 1 1 1 earn . 

'. . \ ' - ■ - . / • '• ;' 

In figure F-l the instructional module is the basic unit of analysis. 
Sdth a module should' be tailored so that it will require an average stoflent 
1 day to master. In. this way, oral and printed formative examinations will 
occur often enough to keep the stu^en:t'^ ;; a|^tention and require a high degree 
of participation. The system i^-tfesigrredVitt keep, the student's mind 'engaged 
'with the. subjedt : matter. - ri Studertf|^||^aiiie<tdy have the skills taught, in, the 
module have trie ability to chaTl;pge : : ,th.e/ cr i ter ion and move *&n to .'other 
instructional modules", if they Can 'demonstrate mastery. 

The process portrayed in the model .is self -correcting. Students pot 
having .either) the prerequisites or . the mastery of the module^ following 
practice do iffot exfi, until they have the criterion ' Skills. 'External 
intervention may be required when' higher' authority determines a given 
student is' spending too much time in a course, arid that the Navy's training 
^tAars.Mqhp be more, effectively spent on someone el sev \ 

" ... e - ,« . . ■'■ • . • ' * ■ - '■ •/ ' % . •'. 

The first decision point in. the. model • involves prerequisites. ' PRQs 
importance ' in the model reflects Bloom's (1976) assertion that it accounts 
1;or 50 percent of student variation in achievement. The basic decisions 
concerning the students' possession of PRQ ca#be made with aptitude %d 
--achievement tests and/or oral examination."' If students do not b.ave the 
prerequisite cognitive and affective behaviors and. if. the instruction itself 
does not adapt to these; .deficiencies, . they 'will probably fail to" : learn /thei'r 
tasks. Thus, the instruction^dapts £o the. individual by (1) branching to a 
task for which, the. student does- have the prerequisites or (2) the 
instructional system itself finding the materials, instructor or tutor 
time, and any other -resources necessary to teach these prerequisites.' '* 

.:•!.' ' . '..'.--"• ^ j ;| : \. . J - V 

Next, the cues are presented' to. enable the students to determine that 
which, they are required- to -learn. As .with other quality of instruction 
variables. CUE does not necessarily have -to follow PRQ or come " at- any ^ 
particular time. As a f-ir^approximation, cues are the terminal learning - 
objectives for the module. —later When information is being, presented, cues , 
aid in learning the .discriminations necessary"- to acquire the concepts or 
other skills involved in the module. .. .;^^vT { .'''.', 

Participation .(PAR) is the focus' of all activity within the model. 
Students learn what they do. Thus, PRQ is ; an attempt' to determine the 
students' readiness for doing something. CUES let ..students know what ' they . 
are to do. FBIC^gives . students i nformatibn as to how-thev are doing" RNF i s 
supposed to strengthen that which students , were doonHT >- And COR provides, 
alternate ways, for 1 students to learn to do thT Tpartloular learning tasks ; 
All of these elements serve tcTkeep the"TAR^OTj^. :}^ I - — ~ 
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Figure. F-l. A, Model of. Quality Individualized Instruction 
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After the . prerequisites are assured, it ;. is the occurrei^ jaojf 
reinforcements that; keep the students 1 attention to the .cues, the ■■4c1f^^-.y- 
the many; -forms of . testing, -and the corrective's^ ; fiigure^F^X 2" showllttNF' . 
occurring at almpst^every point.; With some students, learning is* its °own 
reward. \4: With others, there is. a need for profuse administration'* of 
"attaboys-' or even more material reward. Th(s mo^^^B^^ individualized 
instruction asserts the importance of RNF , at a number ;wP©y 'points, in the 
_act„ofUnstruct-i.on.__ " a ■ -^S— _' •' . 



Feedback can also occur at different stages in the model -through 
different media. Usually, FBK is given' by a test, with additional feedback 
provided by the instructor following the % test./ A test is defined as any 
evaluation of a student's performance. -Thus, FBK occurs during and after 
examinations, in discussion \ with "instructors, and-in rap sessions with -other, 
students ^concerning aspects of the course.. . *' . . 

Correctives (COR) are a form of remedial activity that'are essential *to 
ensure that' all students will learn. When a student^f ails to demonstrate" 
mastery after following a particular instructional sequence, it is incumbent 
on both the instructional materials and^the instructor^ |to determine the 
reason for r failure. '.Once the cause ^f .this failure is diagnosed, . 
alternative materials are prescribed--prWumably, these; materials will, be 
more appropriate: to «the student's specific learning style or skills. Such, 
alternatives might be chapters in- textbooks ' or .technical manuals, 
sound/slide presentations, floppy disks containing mic|bcomputer-!?ased< CAI, 
or- tutoring by an advanced student. ^ f J:. 

: - • . ' " I ■ 

The model is intended as a description of * the essential functions that 
need to be -performed in -good instruction. It is not fior is it intended to 
be a prescription 6f Vhow. these functions are to. be carried out within the 
prerogative of course managers or curriculum developers. Adequate guidance 
for carrying out these functions can, be obtained from the 1 iterature and 
especially NAVEDTRA I10A witn the modifications recommended .in ' section V. 
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